pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Fix backup manifests to generate correct WAL-Ranges across timel
5+ messages / 2 participants
[nested] [flat]

Commits from this thread:
  2021-10-06 ec2133a447 ae254356f9 d6d68e2233  Fix warning in TAP test of pg_verifybackup
* pgsql: Fix backup manifests to generate correct WAL-Ranges across timel
@ 2021-08-23 02:10  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 5+ messages in thread

From: Michael Paquier @ 2021-08-23 02:10 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix backup manifests to generate correct WAL-Ranges across timelines

In a backup manifest, WAL-Ranges stores the range of WAL that is
required for the backup to be valid.  pg_verifybackup would then
internally use pg_waldump for the checks based on this data.

When the timeline where the backup started was more than 1 with a
history file looked at for the manifest data generation, the calculation
of the WAL range for the first timeline to check was incorrect.  The
previous logic used as start LSN the start position of the first
timeline, but it needs to use the start LSN of the backup.  This would
cause failures with pg_verifybackup, or any tools making use of the
backup manifests.

This commit adds a test based on a logic using a self-promoted node,
making it rather cheap.

Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20210818.143031.1867083699202617521.horikyota.ntt@gmail.com
Backpatch-through: 13

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/29f94232517bb5c5f22d1deebecfe2160711150f

Modified Files
--------------
src/backend/replication/backup_manifest.c | 15 +++++++++++----
src/bin/pg_verifybackup/t/007_wal.pl      | 19 ++++++++++++++++++-
2 files changed, 29 insertions(+), 5 deletions(-)



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* pgsql: Fix backup manifests to generate correct WAL-Ranges across timel
@ 2021-08-23 02:10  Michael Paquier <michael@paquier.xyz>
  0 siblings, 1 reply; 5+ messages in thread

From: Michael Paquier @ 2021-08-23 02:10 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix backup manifests to generate correct WAL-Ranges across timelines

In a backup manifest, WAL-Ranges stores the range of WAL that is
required for the backup to be valid.  pg_verifybackup would then
internally use pg_waldump for the checks based on this data.

When the timeline where the backup started was more than 1 with a
history file looked at for the manifest data generation, the calculation
of the WAL range for the first timeline to check was incorrect.  The
previous logic used as start LSN the start position of the first
timeline, but it needs to use the start LSN of the backup.  This would
cause failures with pg_verifybackup, or any tools making use of the
backup manifests.

This commit adds a test based on a logic using a self-promoted node,
making it rather cheap.

Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20210818.143031.1867083699202617521.horikyota.ntt@gmail.com
Backpatch-through: 13

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/a3fcbcda7505e9079cec95e7209cde4f5d5c8bd8

Modified Files
--------------
src/backend/replication/backup_manifest.c | 15 +++++++++++----
src/bin/pg_verifybackup/t/007_wal.pl      | 19 ++++++++++++++++++-
2 files changed, 29 insertions(+), 5 deletions(-)



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* pgsql: Fix backup manifests to generate correct WAL-Ranges across timel
@ 2021-08-23 02:10  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 5+ messages in thread

From: Michael Paquier @ 2021-08-23 02:10 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix backup manifests to generate correct WAL-Ranges across timelines

In a backup manifest, WAL-Ranges stores the range of WAL that is
required for the backup to be valid.  pg_verifybackup would then
internally use pg_waldump for the checks based on this data.

When the timeline where the backup started was more than 1 with a
history file looked at for the manifest data generation, the calculation
of the WAL range for the first timeline to check was incorrect.  The
previous logic used as start LSN the start position of the first
timeline, but it needs to use the start LSN of the backup.  This would
cause failures with pg_verifybackup, or any tools making use of the
backup manifests.

This commit adds a test based on a logic using a self-promoted node,
making it rather cheap.

Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20210818.143031.1867083699202617521.horikyota.ntt@gmail.com
Backpatch-through: 13

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/65b649fecb0c0bd2c2c1f075f94f45a74cdc41be

Modified Files
--------------
src/backend/replication/backup_manifest.c | 15 +++++++++++----
src/bin/pg_verifybackup/t/007_wal.pl      | 19 ++++++++++++++++++-
2 files changed, 29 insertions(+), 5 deletions(-)



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: pgsql: Fix backup manifests to generate correct WAL-Ranges across timel
@ 2021-10-06 02:47  Thomas Munro <thomas.munro@gmail.com>
  parent: Michael Paquier <michael@paquier.xyz>
  0 siblings, 1 reply; 5+ messages in thread

From: Thomas Munro @ 2021-10-06 02:47 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-committers <pgsql-committers@lists.postgresql.org>

On Mon, Aug 23, 2021 at 2:10 PM Michael Paquier <michael@paquier.xyz> wrote:
> Fix backup manifests to generate correct WAL-Ranges across timelines

+$primary->append_conf('standby.signal');

I think this needs a second argument, to silence a warning visible in
regress_log_007_wal.





^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: pgsql: Fix backup manifests to generate correct WAL-Ranges across timel
@ 2021-10-06 03:48  Michael Paquier <michael@paquier.xyz>
  parent: Thomas Munro <thomas.munro@gmail.com>
  0 siblings, 0 replies; 5+ messages in thread

From: Michael Paquier @ 2021-10-06 03:48 UTC (permalink / raw)
  To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-committers <pgsql-committers@lists.postgresql.org>

On Wed, Oct 06, 2021 at 03:47:26PM +1300, Thomas Munro wrote:
> +$primary->append_conf('standby.signal');
> 
> I think this needs a second argument, to silence a warning visible in
> regress_log_007_wal.

Oops, will fix.  Thanks!
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 5+ messages in thread


end of thread, other threads:[~2021-10-06 03:48 UTC | newest]

Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-08-23 02:10 pgsql: Fix backup manifests to generate correct WAL-Ranges across timel Michael Paquier <michael@paquier.xyz>
2021-10-06 02:47 ` Thomas Munro <thomas.munro@gmail.com>
2021-10-06 03:48   ` Michael Paquier <michael@paquier.xyz>
2021-08-23 02:10 pgsql: Fix backup manifests to generate correct WAL-Ranges across timel Michael Paquier <michael@paquier.xyz>
2021-08-23 02:10 pgsql: Fix backup manifests to generate correct WAL-Ranges across timel Michael Paquier <michael@paquier.xyz>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox