pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Avoid truncated or zero-padded copies of replaced files in backu
Date: Tue, 27 Oct 2026 04:56:38 +0000
Message-ID: <E1xLZEk-00000000vo8-0sh1@gemulon.postgresql.org> (raw)

Avoid truncated or zero-padded copies of replaced files in backups

Base backups call stat() on each file while scanning the data directory,
and later send exactly that many bytes.  If the file is replaced by for
example a concurrent rename(), the backup contains the new file
truncated or zero-padded to the size of the old one.  This is harmless
for relation files as WAL replay repairs them.  However, it causes harm
for non-relation files manipulated as temporary file renamed in place.

Here is a list of non-relation files that use a temporary file followed
by a final rename() (I may be missing some):
* postgresql.auto.conf
* pg_logical/replorigin_checkpoint (a damaged copy fails its CRC check
and prevents startup)
* pg_prewarm's autoprewarm.blocks
* pg_stash_advice.tsv.

When sending files that are not relation files, fstat() is called again
on the opened file and the refreshed stats are used for the tar header
and the backup manifest.  Since the rename
is atomic, the opened file is always complete.  Relation files keep using
the earlier stat, to avoid copying data that WAL replay would restore
anyway (because that's cheaper).

Back-patch down to v15.  v14 is going to be EOL soon, so I have decided
to leave it out of the picture (risk/reward thing).

Author: David Steele <david@pgbackrest.org>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Shihao Zhong <zhong950419@gmail.com>
Discussion: https://postgr.es/m/CAGRkXqS_On-pLgcv6xjH8mvGhdcPmfWtythW3r8pYKUG+hsiuQ@mail.gmail.com
Backpatch-through: 15

Branch
------
REL_15_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/1fff28b14af838e3a0e948bfffc98edc02ee5bd3

Modified Files
--------------
src/backend/backup/basebackup.c | 51 +++++++++++++++++++++++++++++------------
1 file changed, 36 insertions(+), 15 deletions(-)



view thread (6+ messages)

Message-ID: <E1xLZEk-00000000vo8-0sh1@gemulon.postgresql.org>
Permalink:  ../E1xLZEk-00000000vo8-0sh1@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1xLZEk-00000000vo8-0sh1@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: michael@paquier.xyz
  Subject: Re: pgsql: Avoid truncated or zero-padded copies of replaced files in backu
  In-Reply-To: <E1xLZEk-00000000vo8-0sh1@gemulon.postgresql.org>
  References: <E1xLZEk-00000000vo8-0sh1@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox