Software & Stuff

Deploy a PostgreSQL Backup Solution with Docker in 5 Minutes

Nikos Andriotis — Wed, 16 Apr 2025 16:42:30 GMT

How to deploy a Postgres continuous backup solution in under 5 minutes using Docker

[Song of the day: HELL ON ME — Johnny Huynh]

Hey there! Its… been a while. I was recently looking to help a collaborator out with backing up a Postgres database and thought it would be a great idea for a blog post. So, let’s begin.

Deployment Method
General Prerequisites
Step 1: Gathering S3-Compatible Bucket Credentials
Step 2: Get your schedule in Cron format
Step 3: (Optional) Gather Sentry Credentials
Step 4: Convert your postgres credentials to a connection URL
Step 5: Filling out the .env file
Step 6: Deploying
Building the image for a version other than 14,15 and 16
GitHub
Regarding stricter permissions for AWS

Deployment Method

There are 2 ways to deploy the backup solution:

Using the Railway Template
Using Docker on your machine / cloud

We will go over each of the 2 step by step to not miss anything

General Prerequisites — Required regardless the method

A Cloudflare / Backblaze / AWS Account
An accessible PostgreSQL database, version 16 to 14
(Optional) A Sentry Account

Step 1: Gathering S3-Compatible Bucket Credentials

Provider 1: AWS

Head over to console.aws.amazon.com and login
Navigate over to the S3 Dashboard in the region of your choice

3. Create or choose a bucket — Note down the bucket name and its region

4. Navigate over to AWS IAM

3. Click on Users then “Create User”

4. Give it the name “pg-backup” or whatever else you’d like

5. Assign the permission “AmazonS3FullAccess”

(Note: As the permission’s name suggests, this grants the user FULL access to ALL s3 buckets, so a more restricted permission set is advised. Check the end of the post for detailed instructions)

5. Proceed to the next page and confirm the user creation.

6. You should now be on the “Users” page. Select the one you just created

7. Navigate over to “Security Credentials”

8. Select “Create Access Key” under “Access Keys”

Choose “Other” on the Use Case
Skip description (or dont?)

9. Save both the Access Key and Secret Access Key

Provider 2: Cloudflare (R2)

Head over to dash.cloudflare.com and login
Choose “R2 Object Storage” from the sidebar

3. Create or choose a bucket and note down its name

4. Click on API > Manage API Tokens (See picture)

5. Create an “Account Token” like below:

Set its name
Change permissions to “Object Read & Write”
Specify your backup bucket
Click “Create”

6. Of the values shown Copy:

The Access Key ID
The Secret Access Key
The Jurisdiction-Specific Endpoint

Provider 3: Backblaze B2

Head over to https://secure.backblaze.com and sign in
Click “Create a Bucket” and choose the following:

Private files
No encryption
No Object Lock

3. From the info of the bucket, note:

The Endpoint
The Bucket name
The Region (found in the endpoint, its first subdomain. For example, the region in the pic is eu-central-003)

4. Now, head over to https://secure.backblaze.com/app_keys.htm

5. Create an Application Key with the following properties

A Name
Access to ONLY the backup bucket
Read & Write scopes

6. Note down the “Key ID” and “Application Key” as “Access Key ID” and “Secret Access Key” respectively

Step 2: Get your schedule in Cron format

Backups run on a cron job. So, we need the schedule for example “Every day at midnight” to become “0 0 * * *”

Your options are:

Use the prompt “cron schedule {schedule}” on ChatGPT
Use crontab.guru

To freshen up your memory on Cron:

It uses the format [second] [minute] [hour] [day_of_month] [month] [day_of_week] , so

* * * * * * runs every second. Also:

* means “Every {minutes / hours / days etc}”
*/a (a is a number) means “Every a {minutes / hours / days etc}”

Step 3: (Optional) Gather Sentry Credentials

If you want to check the status of your backups, you will need Sentry for monitoring. So let’s begin.

Head over to sentry.io and login
If you haven’t yet, create a project by going to the top side of your sidebar, selecting “Projects” then on the top right, “Create new project”
Now, head over to “Crons” then “Add Monitor”

4. Add a name, set it to your desired project, and add the cron schedule from the previous step. You can, of course, mess around with the other settings as well. Then press Create.

5. Copy the monitor slug like shown below and store it:

6. We now need the Sentry DSN:

Go to Settings
Select Projects
Choose the project you used when creating the monitor

Go to Client Keys
Copy and store the DSN

And that’s all for Sentry.

Step 4: Convert your postgres credentials to a connection URL

That’s one of the easy steps. If you already have a connection URL, you can skip this step. Anyway, to convert connection credentials to a URL, use this format:

postgresql://[username]:[password]@[db-host]:[port]/[database]

A note for Supabase users:

The image doesn’t support IPv6. So please use the session pooler instead.

Step 5: Filling out the .env file

You will need to pass the backup config to the container somehow. Here’s how to do that:

Download the .env.preset from GitHub:

curl -o .env "https://raw.githubusercontent.com/andriotisnikos1/pg-backup/refs/heads/main/.env.preset"

2. Fill out the env vars. Let’s go over them one by one:

S3-Compatible ENVs

S3_REGION Is the S3-API compatible region. Fill it with auto if using Cloudflare, or the region that you copied if not using Cloudflare
S3_ENDPOINT Is the endpoint to your bucket. Leave empty if using AWS, use the “ Jurisdiction-Specific Endpoint” if you are using Cloudflare and the bucket “Endpoint” if using Backblaze
S3_AUTH_KEY_ID Is the “Access Key ID”
S3_AUTH_KEY_SECRET Is the “Secret Access Key”
S3_BUCKET is the bucket name

Backups Configuration ENVs

BACKUPS_CRON_SCHEDULE Is the schedule from Step 2
BACKUPS_MAX_KEEP_COUNT (Optional modification, defaults to 5) Is the amount of newest backups to keep before deleting the old ones
BACKUPS_FILE_IDENTIFIER (Optional) Is a string appended to the backup file’s names to distinguish them from others. For example, a configuration with the identifier set to “andriotis” will result in files with naming like so: {date}-{random 32 chars}.andriotis.dump . If left unset, the backup files will follow the {date}-{random 32 chars}.dump convention
BACKUPS_USE_PG_DUMPALL (Optional, defaults to “false”). Activates pg_dumpall instead of pg_dump . Used for bumping entire clusters, instead of a singular DB with its tables and data

Postgres Configuration ENVs

POSTGRES_URL Is the URL constructed at Step 4

(Optional) Sentry Configuration

SENTRY_ENABLED “true” or “false”, enables Sentry. Defaults to “false”
SENTRY_DSN Is the DSN of your project, we copied this at Step 3
SENTRY_MONITOR_SLUG Is the slug of the monitor we created at Step 3

Step 6: Deploying

A note before deploying:

Postgres 16 was released in 2023 and Postgres 15 in 2022. So if your DB is recent, you’re most probably using PG 16. The backup will error out if you’re using an incompatible version so you should probably check it before deploying. You can check the version of your DB by running the query:

select version();

Docker

For Postgres 16:

docker run -d \
    -v "./.env:/app/.env" \
    andriotisnikos1/pg-backup

For Postgres 15

docker run -d \
    -v "./.env:/app/.env" \
    andriotisnikos1/pg-backup:pg-15

For Postgres 14

docker run -d \
    -v "./.env:/app/.env" \
    andriotisnikos1/pg-backup:pg-14

2. Railway

Go to the template and click Deploy Now — Postgres 16 is required!
Input the .env file contents from the previous step
Click deploy

3. Paid Deployment (Hire me!)

If deployment is a hassle, i can do it for you as well as guide you with collecting the right ENV values and answering any of your questions for a coffee (5$). You can find me on fiverr: https://www.fiverr.com/s/BRadeNz

Building the image for a version other than 14,15 and 16

If you want to build for another version, first make sure it supports pg_dump and/or pg_dumpall then:

Clone the repo
Build the image for your version

# clone repo
git clone https://github.com/andriotisnikos1/pg-backup.git

# cd
cd pg-backup

# build
docker build -t pg-backup --build-arg POSTGRES_VERSION=[your-version] .

GitHub

GitHub - andriotisnikos1/pg-backup

Regarding stricter permissions for AWS

Go to the AWS S3 on your desired region
Select your bucket, then click on “Copy ARN”

3. Go back to IAM

4. Click on Policies, then Create Policy

5. Switch to JSON mode and paste in the following:

{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Sid": "Statement1",
   "Effect": "Allow",
   "Action": [
    "s3:ListBucket",
    "s3:PutObject",
    "s3:GetObject",
    "s3:DeleteObject"
   ],
   "Resource": [
    "[ARN from previously]",
    "[ARN from previously]/*"
   ]
  }
 ]
}

Don’t forget to replace your ARN!

This restricts permissions for your user to just the following on your specified bucket:

Retrieving objects
Listing objects
Adding objects
Deleting objects

5. Click Continue, give your policy a name (i named it pg-backup) then click Create policy.

6. Now, when creating your user, you can attach the policy to it, giving it just enough permissions to do its job.

The steps for AWS credentials continue normally for the user creation flow, no need to repeat myself

Making a simple web scraper with an interface using RapidAPI and Next.js

Nikos Andriotis — Fri, 09 Aug 2024 01:45:23 GMT

Wanna scrape? You’ve come to the right place

The demo is publicly available @ scraper-demo.andriotis.me and github.com/andriotisnikos1/scraper-demo

So. You want to scrape some websites but don’t know where to start? Ive got you. I believe you have read the title, so i won’t repeat myself.

What you’re gonna need:

A RapidAPI account
A free subscription to the web scraper and your API Key
Node.js and NPM (Who doesn’t have that IK)
20 minutes of your time

Lets get to it!

Preparation: Getting the RapidAPI API Key

Head over to https://rapidapi.com/developer/dashboard
On the sidebar under “My Apps” select one and head to Authorisation”. If you don’t have a project, create one
Copy your API Key. If you don’t have one, create one with the “RapidAPI” scope

Part 1: Initialising the project

Navigate to your workspace folder. Mine is “~/workspace”

Initialise a project with

npx create-next-app --ts scraper
cd scraper

or if you’re feeling cheeky:

yes | npx create-next-app --ts scraper
cd scraper

this will answer all prompts with “y”, something that we need

Now, add your API Key to .env:

echo "RAPIDAPI_KEY=" > .env

Dependencies

npm i axios sonner @radix-ui/react-icons

Part 2: Server Actions

The API currently supports 2 endpoints, /check and /scrape. Here’s some info about them:

/check Checks if a URL can be scraped, according to the URL’s policies.
/scrape Scrapes a URL, processes it according to the rules

Make the actions.ts file

touch app/actions.ts

2. Add these functions:

"use server" // initialise as server file

import axios from "axios";

// checks if the input URL can be scraped
export async function scrapeCheck(url: string) {
    try {
        const options = {
            headers: {
                'X-RapidAPI-Key': process.env.RAPIDAPI_KEY,
                'X-RapidAPI-Host': 'crawl-and-scrape-check.p.rapidapi.com'
            }
        };
        // build the request URL
        const builtURL = new URL("https://crawl-and-scrape-check.p.rapidapi.com/check")
        builtURL.searchParams.append('url', url)
        builtURL.searchParams.append('agent', 'ScrapeBot') // you can change the value of this
        const res = await axios.get<{ allowed: boolean }>(builtURL.toString(), { headers: options.headers });
        return res.data.allowed
    } catch (error) {
        console.error(error)
        return false
    }
}

// sends scrape request
export async function scrapeURL(url: string) {
    try {
        const options = {
            headers: {
                'X-RapidAPI-Key': process.env.RAPIDAPI_KEY,
                'X-RapidAPI-Host': 'crawl-and-scrape-check.p.rapidapi.com'
            }
        };
        const builtURL = new URL("https://crawl-and-scrape-check.p.rapidapi.com/scrape")
        builtURL.searchParams.append('url', url)
        // you can change this to an element name
        builtURL.searchParams.append('selector', 'body')
        // you can change this to comma-seperated element names
        builtURL.searchParams.append('unwantedFields', 'link,script,head')
        const res = await axios.get<{
            content: string
        }>(builtURL.toString(), { headers: options.headers });
        return res.data.content
    } catch (error) {
        console.error(error)
        return null
    }
}

Part 3: layout.tsx

We are starting to approach the HTML part of this demo. Add these styles to your tag in layout.tsx :

min-h-full flex flex-col w-full items-center justify-center space-y-20

This should be the JSX in layout.tsx :

 
      {children} // change this line

Part 4: Form Actions

In this part, we add the form actions that will run when we press the “Check” and “Scrape” buttons

In your page.tsx file, remove all imports and add this:

"use client"
import { Link2Icon } from "@radix-ui/react-icons";
import React, { useState } from "react";
import { toast, Toaster } from "sonner";
import { scrapeCheck, scrapeURL } from "./actions";

// check url scraping
async function checkURL(f: FormData) {
  // get url
  const inputUrl = f.get("url")
  if (!inputUrl) return toast.error("Please enter a URL", {duration: 5000});
  // scrape check
  toast.loading("Checking if scraping is allowed...", { duration: 5000 });
  const res = await scrapeCheck(inputUrl as string);
  // result
  if (res) toast.success("Scraping is allowed", {duration: 5000});
  else toast.error("Scraping is not allowed", {duration: 5000});
}

// scrape URL
async function scrape(f: FormData) {
  // get url
  const inputUrl = f.get("url");
  if (!inputUrl) return toast.error("Please enter a URL", {duration: 5000});
  // scrape
  toast.loading(`Scraping ${inputUrl} ...`, {duration: 5000});
  const res = await scrapeURL(inputUrl as string);
  if (!res) return toast.error("Failed to scrape the URL", {duration: 5000});
  // download
  const blob = new Blob([res], { type: "text/plain" });
  const url = URL.createObjectURL(blob);
  const a = document.createElement("a");
  a.href = url;
  a.download = `scraped-${f.get("url")!}.html`
  document.body.appendChild(a);
  a.click();
  window.URL.revokeObjectURL(url);
}

Lets talk a bit about these actions:

checkURL takes the URL from the form and outputs the result of the check as a toast
scrape is a bit more interesting. It sends a request for a file scrape then checks if it was successful. In case that’s true, it creates a file and sends it for download!

Part 4: JSX

Finally, replace the JSX part of your page.tsx with this simple form:

export default function Home() {
  const [url, setUrl] = useState("");
  return (
    
      {/* Title */}
      Scrape Check Demo
      
        {/* Main Input */}
        
          
           setUrl(e.target.value)} />
        
        
          {/* Check Button */}
          
            
            
          
          {/* Scrape Button */}
          
            
            
          
        
      
      
    
  );
}

Your page.tsx should look like this:

"use client"
import { Link2Icon } from "@radix-ui/react-icons";
import React, { useState } from "react";
import { toast, Toaster } from "sonner";
import { scrapeCheck, scrapeURL } from "./actions";

// check url scraping
async function checkURL(f: FormData) {
  // get url
  const inputUrl = f.get("url")
  if (!inputUrl) return toast.error("Please enter a URL", {duration: 5000});
  // scrape check
  toast.loading("Checking if scraping is allowed...", { duration: 5000 });
  const res = await scrapeCheck(inputUrl as string);
  // result
  if (res) toast.success("Scraping is allowed", {duration: 5000});
  else toast.error("Scraping is not allowed", {duration: 5000});
}

// scrape URL
async function scrape(f: FormData) {
  // get url
  const inputUrl = f.get("url");
  if (!inputUrl) return toast.error("Please enter a URL", {duration: 5000});
  // scrape
  toast.loading(`Scraping ${inputUrl} ...`, {duration: 5000});
  const res = await scrapeURL(inputUrl as string);
  if (!res) return toast.error("Failed to scrape the URL", {duration: 5000});
  // download
  const blob = new Blob([res], { type: "text/plain" });
  const url = URL.createObjectURL(blob);
  const a = document.createElement("a");
  a.href = url;
  a.download = `scraped-${f.get("url")!}.html`
  document.body.appendChild(a);
  a.click();
  window.URL.revokeObjectURL(url);
}

export default function Home() {
  const [url, setUrl] = useState("");
  return (
    
      {/* Title */}
      Scrape Check Demo
      
        {/* Main Input */}
        
          
           setUrl(e.target.value)} />
        
        
          {/* Check Button */}
          
            
            
          
          {/* Scrape Button */}
          
            
            
          
        
      
      
    
  );
}

This should be the output when you visit localhost:3000:

Reminder: You can also visit scraper-demo.andriotis.me

Now for the last part: Usage

Enter a URL on the field
Click either “Can i scrape that?” or “Scrape URL”

You’ll get your response via toast!

Thanks for reading!

Flexible file chunking for Node.js applications with a mini-sdk

Nikos Andriotis — Sat, 23 Mar 2024 02:35:40 GMT

Introducing Triple Dot

If you’ve happened to see any of my recent tweets, im working on an OSS personal file system that can be controlled by the terminal

So, in order to be able to chunk my files into many, smaller files to upload them progressively and without causing a memory overflow.

We already know that node programs arent the most momory efficient ones and i dont want to make mine a nightmare. Soooooooo i had to build a file chunker.

This is where i built Trple Dot (or “…” or the chunking of a line)

Lets chunk a file together!

Installation

bun install @andriotis/triple-dot

Usage method 1: File System Chunks

import td from "@andriotis/triple-dot";

await td({
    // chunk size in KB
    chunkSize: 1024,
    filePath: "./bun.lockb",
    outputDir: ".triple-dot"
})

after a bit, .triple-dot will contain files with the format #.chunk where # is the number of the chunk

Before we move on to the second method, id like to mention the function getFileChunkStats that allows you to predict the behaviour of triple dot by providing you with the chunk amount it will generate for a given file as well as its stats from fs.

Usage

import { getFileChunkStats } from "@andriotis/triple-dot";

const stats = await getFileChunkStats("bun.lockb", 1024);
console.log(stats);

Output

{
  "chunks": 1 (or another number),
  "stats": ... (typeof fs.Stats)
}

Usage method 2: Callback Function Chunks

Triple Dot has the option to pass the chunk buffer to an onProgress function instead of writing to the file system. Lets see how this works

import td from "@andriotis/triple-dot";

const outputToProgress = true;

await td({
  chunkSize: 1024,
  filePath: "bun.lockb",
  outputDir: ".chunks",
  passChunkToCallback: outputToProgress,
  onProgress: (chunkNumber, totalChunks, buffer) => {
    // do stuff with the buffer....
    return;
  },
});

And that’s how you can chunk a file with triple dot! If you have any questions, feel free to email me at medium@andriotis.me

Oh, i almost forgot. Triple dot is OSS. You can check it out here: https://github.com/andriotisnikos1/file-chunker

thats all for today. adios!

Polyhook Product Hunt launch report.

Nikos Andriotis — Fri, 16 Feb 2024 21:51:42 GMT

TLDR of the good stuff, the bad stuff and the lessons

Launching a product or tool on PH is always something that makes you anxious, and I was no exception. In summary, the tool generated little to no attention getting 23 upvotes in 13 hours.

The good things:

i got back into development.
i got some, although little, traffic from twitter
5 people signed up.
20 projects were created.

The bad stuff:

Polyhook wasnt mobile friendly (I KNOW, i know)
I only had github as method of oauth. i cringe every time i think of this
Signup rate was very low (2%)

Lessons:

Even if you’re targeting developers. Google OAuth is still a popular choice among them. Don’t just go with GitHub. Otherwise, add Email signin
Make the app mobile friendly by design and from the first edition. Its not something dark mode. responsiveness is necessary. fun fact: roughly 3/5 user agents were indicating a mobile device. take advantage of that.
The fact that you’re solving a problem, doesn’t mean that it needs solving. sure, some people will want to use a webhook invoker. invoking webhooks is a trivial matter though. the 2% signup rate is more than justified.
Product Hunt isnt the first and/or last place you should launch. It requires a small existing audience to support you. I suggest listing your product in directories first. after that, go for the big bucks.

That’s all. Have a nice day!

PS. song recommendation: “Goodbye” — Ramsey

Instantly share files and code, with anyone from your terminal.

Nikos Andriotis — Fri, 13 Oct 2023 14:06:11 GMT

No joke. Its name is ICS.

“Hey Nikos, can you send me hello.c?” That was the question a friend asked me that caused the discovery of this dev tool.

No need for Discord, Dropbox, Drive, permissions, share links and browser tabs.

Amazing, isn’t it? Lets see how this works:

Before we begin, here is the npm link to the package

1. Installation

npm i -g ics-cli

2. Username reservation

This will allow you to use ICS. otherwise, shares were gonna be a mess.

ics username reserve

Aaaaand thats it lmao. now, lets create a share

3. Create a share

a. Preparation: a temporary file

cd ~/Downloads
echo "hello there" > hello.txt

b. Publish

ics share create

Follow the instructions given to you by the CLI. The output will look something like this:

As you can see, i now have a single use share called test on my username nikos-medium . For multi-use shares, use thepermanent flag. For other commands look no further than

ics teachme
#OR
ics getting_started
#OR
ics  --help

4. Accessing Shares

If you have sufficient permissions, no matter your username, you can access an ICS Share via

ics share get USERNAME/SHARE_NAME
#Example
ics share get nikos-medium/test

5. Deleting Shares

ics share delete

6. Small video demo

https://medium.com/media/b5a136fe3aad394bca4025be1a19932e/href

Consider following me on Twitter: https://twitter.com/andriotisnikos1

Save up to 216$/y/domain on your GCP bill by connecting Cloud Run to Firebase

Nikos Andriotis — Sat, 09 Sep 2023 22:10:50 GMT

No bs. Εxplanation included.

Note: See at the end of post for savings breakdown.

Let’s connect firebase with Cloud Run!

Requirements

Node.js @ latest

Step 1: Install the firebase CLI

npm i -g firebase-tools

Step 2: Ensure you are logged in to google via browser.

Step 3: Create a Firebase project

Go to http://console.firebase.google.com/
Click Add Project
Enter a name and click Continue
Keep Google Analytics enabled and click Continue
Choose or create a Google Analytics account or select an existing one.
Click Create Project
Wait for the process to finish and click Continue. You’ll be redirected to the project dashboard

Step 4: Enable Pay-as-you-go

We need to enable Pay-as-you-go, so our custom domains don’t shut down! See more on the cost breakdown below!

Go to the bottom left corner of the dashboard. Click Upgrade , next to Spark
Select or create a GCP billing account. Its easy to create if you don’t already have one
Click Continue then Purchase

Pay as you go is now enabled!

Step 5: Deploy your apps to Cloud Run

For help, you can refer to a similar tutorial I made, up until step 7

Step 6: Get your firebase project ID

On the firebase dashboard:

click the Gear Icon on the top left, on the right of Project Overview
go to Project Settings > General > Your Project
Copy the Project ID value and store it somewhere.

Step 8: Create hosting sites

On the firebase dashboard:

click Hosting on the top left, under Project Shortcuts
For each of your apps you want to deploy, click the Add another site button on the right of your screen, and give it a Unique Name (unique_name.web.app)
Copy each Unique Name and store it somewhere

Step 8: firebase.json

Go somewhere in your desktop (eg downloads/temp) and create a firebase.json file with the following

{
    "$schema": "https://raw.githubusercontent.com/mainrs/vscode-firebase-configuration-schema-plugin/master/schemas/firebase.schema.json",
    "hosting": []
}

We’re gonna fill out the hosting array

Step 9: Login to firebase via the CLI

firebase login

Step 10: Create hosting targets for the unique names

For each of the Unique Names you created, apply a hosting target alias by running:

firebase target:apply hosting

Step 11: Fill the hosting array

Head over to https://console.cloud.google.com/run
Use this page as a reference to the names and regions of your Cloud Run services

The schema for a hosting element

{
  "target": "",
  "rewrites": [
    {
      "source": "/*",
      "run": {
        "serviceId": "",
        "region": "",
        "pinTag": true
      }
    }
  ]
}

Example

Cloud Run Service Name: api
Cloud Run Service Region: us-central1 (Iowa)
Firebase Hosting Site Unique Name: api-ybbe34t3

I run:

firebase target:apply hosting cr-api api-ybbe34t3

Then the schema:

{
  "target": "cr-api",
  "rewrites": [
    {
      "source": "/*",
      "run": {
        "serviceId": "api",
        "region": "us-central1",
        "pinTag": true
      }
    }
  ]
}

Step 12: Deploy!

firebase deploy --only hosting --project

Step 13: The custom domains.

head over to the firebase console
look top-left on the sidebar > “Hosting”
click “View” on one of your sites
in the middle of the screen, you will see a “Add custom domain” button. Click it
Insert the domain you want to use, click Continue
Change the DNS records of your domain accordingly

That's it!

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Cost Breakdown

Load Balancing on GCP costs:

261$/y for the first 5 rules (0.025$/hr)
86,4$/y for every other one (0.01$/hr)

There is no cost for Networking

Hosting on Firebase costs:

15 cents / GB of Ingress/Egress
Some other costs that don’t apply (BC of Cloud Run)

For a small company with 1TB Networking and 5 Cloud Run services, in a year:

5 * 261$ = 1305$ for GCP
1024 * 0,15$ = 130,6$ for Firebase!

When should I move to GCP?

When your cost of Networking/ hr exceeds your GCP LB costs/hr

Deploy to GCP Cloud Run using GitHub Actions Workflows.

Nikos Andriotis — Wed, 06 Sep 2023 18:59:15 GMT

No bs. TLDR included.

Lets get started. Requirements for today are:

GitHub repo page to your repo
https://console.cloud.google.com/iam-admin/serviceaccounts
your repo open in your preferred editor
a Dockerfile in your repo with build instructions

Note: Anything marked with “< >”(eg ) must be changed to reflect your repo/choice

Step 1: Create the workflow file

Head over to your repo in your editor
create a file under .github/workflows called .yml. the .github/workflows directory must be in the root of your project

example: .github/workflows/cloud_run.yml

Step 2: start editing your file

Step 3: paste the following code in your file

This code was created based on this page and the gcp auth workflow repo

on:
    pull_request: #you can also setup your own triggers
        branches: 
            - main

jobs:
    gc-cloud_run:
        name: 'cloud_run'
        runs-on: ubuntu-latest

        permissions:
            contents: 'read'
            deployments: 'write'
            id-token: 'write'

        steps:
            - uses: 'actions/checkout@v3'

            - id: 'auth'
              name: 'Gcloud Auth'
              uses: 'google-github-actions/auth@v1'
              with:
                credentials_json: '${{secrets.GCLOUD_AUTH}}'
  
            #you can replace "us-central1" with your preferred region
            - run: |-
                gcloud run deploy  --platform managed --source  --region us-central1;

Step 4: Publish your changes to GitHub

Step 5: IAM service account

head over to https://console.cloud.google.com/iam-admin/serviceaccounts
click create service account at the top of the page

add a service account name, let the account ID be generated automatically
press create and continue . Dont skip step the other steps
click the dropdown: select a role
find “Editor” under Quick Access > Basic > Editor
click continue
click done

Step 6: Service account credentials

head back to https://console.cloud.google.com/iam-admin/serviceaccounts
click on your service account
navigate to the KEYS tab
click “Add Key” > “Create new Key”
select JSON format and Create . a key file will be downloaded
go to your downloads and copy the contents of the key file

Step 7: create the `GCLOUD_AUTH` env

go to your github repo page
navigate to the “Settings” tab
on the sidebar, look under “Security” to find “Secrets and Variables” > “Actions”
click on “New repository secret”
name it “GCLOUD_AUTH”
paste in the contents of the key file

You're done! Now, if the correct conditions are met, the action will be triggered, deploying your app to cloud run!

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

TLDR

create a new workflow file with the contents of Step 3, replacing the required values
create a new GCP service account with ‘Editor’ permissions
get json credentials for service account
create an env GCLOUD_AUTH with the json credentials as contents

Deploy a TS server/client using Cloud Run and Cloudflare

Nikos Andriotis — Mon, 04 Sep 2023 18:44:44 GMT

No BS. The sauce.

Reference: My package.json scripts

"scripts": {
    "dev": "nodemon --watch 'src/**/*.ts' --exec 'node --loader ts-node/esm' src/index.ts",
    "start": "node --loader ts-node/esm ./src/index.ts"
 }

Step 1: Confirm you’re logged in to Google from your browser.

Step 2: Make sure` gcloud` CLI is installed

brew:

brew install google-cloud-sdk --cask

scoop

scoop bucket add extras
scoop install extras/gcloud

Step 3: Login with gcloud CLI

gcloud auth login

Navigate to the given URL and login with your Google Account

Note: find all your projectIDs here, top left corner, to the right of the logo: https://console.cloud.google.com/welcome

Step 4: navigate to your project locally

cd path/to/project

Step 5: tsconfig.json

You need rootDir and outDir . Other than that, do as you wish. Here is mine:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "ES2022",
    "rootDir": "./src",
    "moduleResolution": "node",
    "outDir": "./dist",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true
  }
}

Step 6: Dockerfile

Customise it as you wish. Here is mine:

FROM node:20.5.1-alpine

WORKDIR /app

COPY package.json .

RUN npm install

COPY . .

run npx tsc

CMD node ./dist

Step 7: Launch

From your root project dir:

gcloud run deploy --platform managed --set-env-vars NODE_ENV=production

After that, configure your deployment settings as you wish. Here are mine:

Allow Unauthenticated Invocations: true
Data Center: lowa (number 30 on the list)

Step 8: Custom domain setup: GCP

head over to https://console.cloud.google.com/run/
click on your deployed app
go to Integrations

click Add Intgration
select Custom Domains — Google Cloud Load Balancing
under Routes replace Domain 1 with your custom domain eg. hello.example.com
scroll to page bottom and click Deploy .

Step 9: Get the Load Balancer IP

wait for 2–5 minutes
refresh the GCP page and click View Details on the integration.

scroll to the bottom
you should now see the details for an A DNS Record. copy the IP/ record value

Step 10: Cloudflare

head over to the Cloudflare Dashboard
login
go to your website
add a DNS record of type A and paste the IP of the Load Balancer you copied on step 9.
set the record type to proxied

You’re done. Now, wait for the Google CA to issue your SSL cert and you’ll be good to go!

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Deleting the integration

GCP does not allow you to delete the integration (error, idk). Here is a bypass:

Method 1: delete the cloud run app

Method 2: delete all integration resources

head over to https://console.cloud.google.com/run/
click on your deployed app
go to Integrations

click View Details on the integration
scroll down to Resources
go one-by-one and delete everything

After you’re done, just delete the DNS record from Cloudflare and that's it!

Software & Stuff

Deploy a PostgreSQL Backup Solution with Docker in 5 Minutes

How to deploy a Postgres continuous backup solution in under 5 minutes using Docker

Table of contents

Deployment Method

General Prerequisites — Required regardless the method

Step 1: Gathering S3-Compatible Bucket Credentials

Provider 1: AWS

Provider 2: Cloudflare (R2)

Provider 3: Backblaze B2

Step 2: Get your schedule in Cron format

Step 3: (Optional) Gather Sentry Credentials

Step 4: Convert your postgres credentials to a connection URL

A note for Supabase users:

Step 5: Filling out the .env file

Step 6: Deploying

Building the image for a version other than 14,15 and 16

GitHub

Regarding stricter permissions for AWS

Making a simple web scraper with an interface using RapidAPI and Next.js

Preparation: Getting the RapidAPI API Key

Part 1: Initialising the project

Part 2: Server Actions

Part 3: layout.tsx

Part 4: Form Actions

Part 4: JSX

Scrape Check Demo

Scrape Check Demo

Now for the last part: Usage

Flexible file chunking for Node.js applications with a mini-sdk

Installation

Usage method 1: File System Chunks

Usage method 2: Callback Function Chunks

Polyhook Product Hunt launch report.

Instantly share files and code, with anyone from your terminal.

1. Installation

2. Username reservation

3. Create a share

a. Preparation: a temporary file

b. Publish

4. Accessing Shares

5. Deleting Shares

6. Small video demo

Save up to 216$/y/domain on your GCP bill by connecting Cloud Run to Firebase

Deploy to GCP Cloud Run using GitHub Actions Workflows.

TLDR

Deploy a TS server/client using Cloud Run and Cloudflare

Deleting the integration