public inbox for [email protected]  

help / color / mirror / Atom feed

From: David Steele <[email protected]>
To: Pg Hackers <[email protected]>
Subject: pg_combinebackup does not detect missing files
Date: Thu, 11 Apr 2024 11:36:30 +1000
Message-ID: <[email protected]> (raw)

Hackers,

I've been playing around with the incremental backup feature trying to 
get a sense of how it can be practically used. One of the first things I 
always try is to delete random files and see what happens.

You can delete pretty much anything you want from the most recent 
incremental backup (not the manifest) and it will not be detected.

For example:

$ pg_basebackup -c fast -D test/backup/full -F plain
$ pg_basebackup -c fast -D test/backup/incr1 -F plain -i 
/home/dev/test/backup/full/backup_manifest

$ rm test/backup/incr1/base/1/INCREMENTAL.2675
$ rm test/backup/incr1/base/1/826
$ /home/dev/test/pg/bin/pg_combinebackup test/backup/full 
test/backup/incr1 -o test/backup/combine

$ ls test/backup/incr1/base/1/2675
No such file or directory
$ ls test/backup/incr1/base/1/826
No such file or directory

I can certainly use verify to check the backups individually:

$ /home/dev/test/pg/bin/pg_verifybackup /home/dev/test/backup/incr1
pg_verifybackup: error: "base/1/INCREMENTAL.2675" is present in the 
manifest but not on disk
pg_verifybackup: error: "base/1/826" is present in the manifest but not 
on disk

But I can't detect this issue if I use verify on the combined backup:

$ /home/dev/test/pg/bin/pg_verifybackup /home/dev/test/backup/combine
backup successfully verified

Maybe the answer here is to update the docs to specify that 
pg_verifybackup should be run on all backup directories before 
pg_combinebackup is run. Right now that is not at all clear.

In fact the docs say, "pg_combinebackup will attempt to verify that the 
backups you specify form a legal backup chain". Which I guess just means 
the chain is verified and not the files, but it seems easy to misinterpret.

Overall I think it is an issue that the combine is being driven from the 
most recent incremental directory rather than from the manifest.

Regards,
-David

view thread (2+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected]
  Subject: Re: pg_combinebackup does not detect missing files
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox