From: Robert Haas <rhaas@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Add support for incremental backup.
Date: Wed, 20 Dec 2023 14:54:44 +0000
Message-ID: <E1rFxy7-00B5yI-K1@gemulon.postgresql.org> (raw)
Add support for incremental backup.
To take an incremental backup, you use the new replication command
UPLOAD_MANIFEST to upload the manifest for the prior backup. This
prior backup could either be a full backup or another incremental
backup. You then use BASE_BACKUP with the INCREMENTAL option to take
the backup. pg_basebackup now has an --incremental=PATH_TO_MANIFEST
option to trigger this behavior.
An incremental backup is like a regular full backup except that
some relation files are replaced with files with names like
INCREMENTAL.${ORIGINAL_NAME}, and the backup_label file contains
additional lines identifying it as an incremental backup. The new
pg_combinebackup tool can be used to reconstruct a data directory
from a full backup and a series of incremental backups.
Patch by me. Reviewed by Matthias van de Meent, Dilip Kumar, Jakub
Wartak, Peter Eisentraut, and Álvaro Herrera. Thanks especially to
Jakub for incredibly helpful and extensive testing.
Discussion: http://postgr.es/m/CA+TgmoYOYZfMCyOXFyC-P+-mdrZqm5pP2N7S-r0z3_402h9rsA@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/dc212340058b4e7ecfc5a7a81ec50e7a207bf288
Modified Files
--------------
doc/src/sgml/backup.sgml | 89 +-
doc/src/sgml/config.sgml | 2 -
doc/src/sgml/protocol.sgml | 24 +
doc/src/sgml/ref/allfiles.sgml | 1 +
doc/src/sgml/ref/pg_basebackup.sgml | 37 +-
doc/src/sgml/ref/pg_combinebackup.sgml | 240 ++++
doc/src/sgml/reference.sgml | 1 +
src/backend/access/transam/xlogbackup.c | 10 +
src/backend/access/transam/xlogrecovery.c | 6 +
src/backend/backup/Makefile | 1 +
src/backend/backup/basebackup.c | 319 ++++-
src/backend/backup/basebackup_incremental.c | 1003 ++++++++++++++++
src/backend/backup/meson.build | 1 +
src/backend/replication/repl_gram.y | 14 +-
src/backend/replication/repl_scanner.l | 2 +
src/backend/replication/walsender.c | 162 ++-
src/backend/storage/ipc/ipci.c | 3 +
src/bin/Makefile | 1 +
src/bin/meson.build | 1 +
src/bin/pg_basebackup/bbstreamer_file.c | 1 +
src/bin/pg_basebackup/pg_basebackup.c | 112 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/bin/pg_combinebackup/.gitignore | 1 +
src/bin/pg_combinebackup/Makefile | 52 +
src/bin/pg_combinebackup/backup_label.c | 283 +++++
src/bin/pg_combinebackup/backup_label.h | 30 +
src/bin/pg_combinebackup/copy_file.c | 169 +++
src/bin/pg_combinebackup/copy_file.h | 19 +
src/bin/pg_combinebackup/load_manifest.c | 245 ++++
src/bin/pg_combinebackup/load_manifest.h | 67 ++
src/bin/pg_combinebackup/meson.build | 38 +
src/bin/pg_combinebackup/nls.mk | 11 +
src/bin/pg_combinebackup/pg_combinebackup.c | 1284 +++++++++++++++++++++
src/bin/pg_combinebackup/reconstruct.c | 687 +++++++++++
src/bin/pg_combinebackup/reconstruct.h | 33 +
src/bin/pg_combinebackup/t/001_basic.pl | 23 +
src/bin/pg_combinebackup/t/002_compare_backups.pl | 154 +++
src/bin/pg_combinebackup/t/003_timeline.pl | 90 ++
src/bin/pg_combinebackup/t/004_manifest.pl | 75 ++
src/bin/pg_combinebackup/t/005_integrity.pl | 125 ++
src/bin/pg_combinebackup/write_manifest.c | 293 +++++
src/bin/pg_combinebackup/write_manifest.h | 33 +
src/bin/pg_resetwal/pg_resetwal.c | 36 +
src/include/access/xlogbackup.h | 2 +
src/include/backup/basebackup.h | 5 +-
src/include/backup/basebackup_incremental.h | 55 +
src/include/nodes/replnodes.h | 9 +
src/test/perl/PostgreSQL/Test/Cluster.pm | 21 +-
src/tools/pgindent/typedefs.list | 12 +
49 files changed, 5834 insertions(+), 52 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: rhaas@postgresql.org, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Add support for incremental backup.
In-Reply-To: <E1rFxy7-00B5yI-K1@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox