Subject: pg_basebackup --incremental

From: Sam Stearns <[email-masked]>
Date: Tue, 28 Oct 2025 10:42:45 -0700

Howdy,

We're running version 17.6.  Would anyone be able to point me to, or
provide, some sample use cases / scripts / usage to deploy a pg_basebackup
full + --incremental strategy as a backup solution, please?

This will be used temporarily as we are migrating to RDS soon which is why
we're not going with pgBackRest.

Thanks,

Sam

-- 

Samuel Stearns
Team Lead - Database
c: 971 762 6879 | o: 971 762 6879 | DAT.com

<https://www.dat.com/?utm_medium=email&utm_source=DAT_email_signature_link>

---

From: Ron Johnson <[email-masked]>
Date: Tue, 28 Oct 2025 13:52:08 -0400

> full + --incremental strategy as a backup solution, please?

The question confuses me a bit (though maybe because weekly "full", and
remainder "incremental" is pretty standard).  PgBackRest really is quite
simple and easy to configure if you back up to a local mount point (even
when that mount point is NFS).

This is in the "postgres" crontab:
15 01 * * Sun Type=full; pgbackrest backup --stanza=nfs --type=$Type &>
logs/pgbackrest_$(date +"\%F_\%T")_${Type}.log
15 01 * * 1-6 Type=incr; pgbackrest backup --stanza=nfs --type=$Type &>
logs/pgbackrest_$(date +"\%F_\%T")_${Type}.log

And this is my /etc/pgbackrest.conf:
[global]
repo1-path=/Database/backups/pgbackrest
repo1-cipher-type=aes-256-cbc
repo1-cipher-pass=<redacted>
repo1-bundle=y
repo1-bundle-limit=20MiB
repo1-bundle-size=200MiB
[nfs]
pg1-path=/Database/17/data
resume=n
start-fast=y
stop-auto=y
compress-type=zst
log-level-console=detail
log-level-file=info
log-path=/var/lib/pgsql/logs/pgbackrest
retention-full=4
process-max=<nproc * 3/4>
[nfs:archive-push]
compress-type=zst

-- 
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

---

From: Sam Stearns <[email-masked]>
Date: Tue, 28 Oct 2025 15:21:01 -0700

Thanks, Ron!  We'll take another look at pgBackRest.

On Tue, Oct 28, 2025 at 10:52 AM Ron Johnson <[email-masked]>
wrote:

> On Tue, Oct 28, 2025 at 1: 43 PM Sam Stearns <sam. stearns@ dat. com>
> wrote: Howdy, We're running version 17. 6. Would anyone be able to point me
> to, or provide, some sample use cases / scripts / usage to deploy a
> pg_basebackup full + --incremental
> ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
>
> ZjQcmQRYFpfptBannerEnd

-- 

Samuel Stearns
Team Lead - Database

---

From: Sam Stearns <[email-masked]>
Date: Wed, 29 Oct 2025 10:40:28 -0700

Hi Ron,

If I may, please.  What are the postgres.conf parameters you set
specifically for pgBackRest?

Thanks,

Sam

-- 

Samuel Stearns
Team Lead - Database

---

From: Ron Johnson <[email-masked]>
Date: Wed, 29 Oct 2025 13:52:55 -0400

I'm certain this is all in the User Guide:
$ grep -rh archive $PGDATA/postgresql.conf*
archive_mode = on
#archive_command = '/bin/true'
archive_command = 'pgbackrest --stanza=nfs archive-push %p'

Since changing archive_mode requires a restart, but
changing archive_command just requires a reload, it's useful to have both
of those archive_command lines in your config.  Always keep "archive_mode =
on" but disable it by setting archive_command to /bin/true (which will be a
rare occurrence).

---

From: Sam Stearns <[email-masked]>
Date: Wed, 29 Oct 2025 11:26:30 -0700

Awesome.  Thank you, Ron.

We have our archive_command set to push WAL's to standby as such:

archive_command = 'test ! -f /postgres_wal_archive/rtsstage/%f && rsync -a
%p /postgres_wal_archive/rtsstage/%f && rsync -a %p [email-masked]:
/postgres_wal_archive/rtsstage/%f'

How does incorporating pgbackrest affect that setup?

On Wed, Oct 29, 2025 at 10:53 AM Ron Johnson <[email-masked]>
wrote:

> I'm certain this is all in the User Guide: $ grep -rh archive
> $PGDATA/postgresql. conf* archive_mode = on #archive_command = '/bin/true'
> archive_command = 'pgbackrest --stanza=nfs archive-push %p' Since changing
> archive_mode
> ZjQcmQRYFpfptBannerStart

-- 

Samuel Stearns
Team Lead - Database

---

From: Ron Johnson <[email-masked]>
Date: Wed, 29 Oct 2025 15:12:01 -0400

pgbackrest does WAL archiving for you, as seen here:
$ dir /Database/backups/pgbackrest/
total 64
drwxr-x--- 3 nobody nobody 21 2024-06-08 21:35:11 archive/
drwxr-x--- 3 nobody nobody 21 2025-06-11 13:04:10 backup/

Thus, it obviates the need for the rsync commands.

If, like me, you still want the WAL files to *also* be somewhere else, then
create a cron job to rsync {repo1-path}/pgbackrest/archive to <insert
destination here> every15 minutes or so. Add "--del" to remove the WALs
that pgbackrest deletes when it purges the oldest saveset.

On Wed, Oct 29, 2025 at 2:26 PM Sam Stearns <[email-masked]> wrote:

>>>>> The question confuses me a bit (though maybe because weekly "full",
>>>>> and remainder "incremental" is pretty standard).  PgBackRest really is
>>>>> quite simple and easy to configure if you back up to a local mount point
>>>>> (even when that mount point is NFS).
>>>>>
>>>>> 15 01 * * Sun Type=full; pgbackrest backup --stanza=nfs --type=$Type
>>>>> &> logs/pgbackrest_$(date +"\%F_\%T")_${Type}.log
>>>>> 15 01 * * 1-6 Type=incr; pgbackrest backup --stanza=nfs --type=$Type
>>>>> &> logs/pgbackrest_$(date +"\%F_\%T")_${Type}.log
>>>>>

---

From: Sam Stearns <[email-masked]>
Date: Wed, 29 Oct 2025 12:19:25 -0700

Thanks, Ron.  Very helpful.

On Wed, Oct 29, 2025 at 12:12 PM Ron Johnson <[email-masked]>
wrote:

> pgbackrest does WAL archiving for you, as seen here: $ dir
> /Database/backups/pgbackrest/ total 64 drwxr-x--- 3 nobody nobody 21
> 2024-06-08 21: 35: 11 archive/ drwxr-x--- 3 nobody nobody 21 2025-06-11 13:
> 04: 10 backup/ Thus, it obviates the need
> ZjQcmQRYFpfptBannerStart
> If, like me, you still want the WAL files to *also* be somewhere else,
> then create a cron job to rsync {repo1-path}/pgbackrest/archive to <insert
>> archive_command = 'test ! -f /postgres_wal_archive/rtsstage/%f && rsync
>> -a %p /postgres_wal_archive/rtsstage/%f && rsync -a %p
>> [email-masked]:/postgres_wal_archive/rtsstage/%f'
>>
>>>>>> On Tue, Oct 28, 2025 at 1: 43 PM Sam Stearns <sam. stearns@ dat.
>>>>>> com> wrote: Howdy, We're running version 17. 6. Would anyone be able to
>>>>>> point me to, or provide, some sample use cases / scripts / usage to deploy
>>>>>> a pg_basebackup full + --incremental
>>>>>> ZjQcmQRYFpfptBannerStart

-- 

Samuel Stearns
Team Lead - Database

---

From: Serkan Akdemir <[email-masked]>
Date: Fri, 31 Oct 2025 10:28:00 +0000

I created a script for incremental backup+merge on PostgreSQL 17+
It creates a full backup and later (next run) incremental backups. Incremental backups are merged to full backup with script, so you have always updated full backup.
Disclaimer: use it at your own risk.

Simple usage is:
# first backup should be full
pg_backup.sh --mode=fulll

# then inc
pg_backup.sh --mode=fincremental

<https://github.com/rdnkrkmz/medium-resources/blob/main/pg_backup.sh>
medium-resources/pg_backup.sh at main · rdnkrkmz/medium-resources<https://github.com/rdnkrkmz/medium-resources/blob/main/pg_backup.sh>
github.com<https://github.com/rdnkrkmz/medium-resources/blob/main/pg_backup.sh>
<https://github.com/rdnkrkmz/medium-resources/blob/main/pg_backup.sh>

On 28 Oct 2025, at 20:42, Sam Stearns <[email-masked]> wrote:

Howdy,

We're running version 17.6.  Would anyone be able to point me to, or provide, some sample use cases / scripts / usage to deploy a pg_basebackup full + --incremental strategy as a backup solution, please?

This will be used temporarily as we are migrating to RDS soon which is why we're not going with pgBackRest.

Thanks,

Sam

--

Samuel Stearns
Team Lead - Database

[https://lh7-rt.googleusercontent.com/docsz/AD_4nXchMTXZiU3P-OkN7UwPyrdxiCAOkHWIArmSrehhAl-cXaYxECakbMV5owUfMaMjAZZcbjttr8pEtSGrZjUSRNaxbLxumAthLgYlFE5iT2P2eOJu4jtqizp_fYM0Mq1DZyFUeHMhpGKjiGIzqD_ny_hPIzlt?key=z3dM0-Cr9WzKyLauPVBgWw]<https://www.dat.com/?utm_medium=email&utm_source=DAT_email_signature_link>

---

From: Sam Stearns <[email-masked]>
Date: Fri, 31 Oct 2025 07:25:34 -0700

Thanks very much, Serkan.  We'll have a look.

On Fri, Oct 31, 2025 at 3:28 AM Serkan Akdemir <[email-masked]> wrote:

> I created a script for incremental backup+merge on PostgreSQL 17+ It
> creates a full backup and later (next run) incremental backups. Incremental
> backups are merged to full backup with script, so you have always updated
> full backup. Disclaimer:
> ZjQcmQRYFpfptBannerStart
> This Message Is From an Untrusted Sender
> You have not previously corresponded with this sender.
>
> ZjQcmQRYFpfptBannerEnd
> It creates a full backup and later (next run) incremental backups.
> Incremental backups are merged to full backup with script, so you have
> always updated full backup.
> medium-resources/pg_backup.sh at main · rdnkrkmz/medium-resources
> github.com

-- 

Samuel Stearns
Team Lead - Database

---

From: Ron Johnson <[email-masked]>
Date: Fri, 31 Oct 2025 11:16:50 -0400

Note that pg_basebackup is single-threaded, and not encrypted.

---

From: Sam Stearns <[email-masked]>
Date: Fri, 31 Oct 2025 08:33:40 -0700

Thanks, Ron.

On Fri, Oct 31, 2025 at 8:17 AM Ron Johnson <[email-masked]> wrote:

> Note that pg_basebackup is single-threaded, and not encrypted. On Fri, Oct
> 31, 2025 at 10: 26 AM Sam Stearns <sam. stearns@ dat. com> wrote: Thanks
> very much, Serkan. We'll have a look. On Fri, Oct 31, 2025 at 3: 28 AM
> Serkan Akdemir <osmosyum@ hotmail. com>
> ZjQcmQRYFpfptBannerStart

-- 

Samuel Stearns
Team Lead - Database