pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Fujii Masao <masao.fujii@oss.nttdata.com>
To: Robert Haas <robertmhaas@gmail.com>
To: David Steele <david@pgmasters.net>
Cc: Andres Freund <andres@anarazel.de>
Cc: Noah Misch <noah@leadboat.com>
Cc: Stephen Frost <sfrost@snowman.net>
Cc: Amit Kapila <amit.kapila16@gmail.com>
Cc: Suraj Kharage <suraj.kharage@enterprisedb.com>
Cc: tushar <tushar.ahuja@enterprisedb.com>
Cc: Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>
Cc: Rushabh Lathia <rushabh.lathia@gmail.com>
Cc: Tels <nospam-pg-abuse@bloodgate.com>
Cc: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Cc: Jeevan Chalke <jeevan.chalke@enterprisedb.com>
Cc: vignesh C <vignesh21@gmail.com>
Subject: Re: backup manifests
Date: Wed, 8 Apr 2020 14:15:38 +0900
Message-ID: <78f76a3d-1a28-a97d-0394-5c96985dd1c0@oss.nttdata.com> (raw)
In-Reply-To: <CA+TgmoaKhxPr0FWsEiZoXpMER44pMJ28L-Z7wcRrOKoGFwjiUQ@mail.gmail.com>
References: <20200330185944.42bxysvem6b757ew@alap3.anarazel.de>
	<CA+TgmoawEeE5qpFgj5Vy2zZGKzd3ZSEhGrD_JdPqPd2GB8u1Cw@mail.gmail.com>
	<20200331225034.odt3l2w4h3dr6ggk@alap3.anarazel.de>
	<CA+TgmobV+kW5cBCa8_LGvytdOH71gnpqDsmKBR7bghEFgR7Y1w@mail.gmail.com>
	<CA+Tgmob+xDcvEUTznkvianyJzKK8nUM6bWfkrnZQkL-XEa3NEA@mail.gmail.com>
	<20200402172318.3kvfrrewxmemzpia@alap3.anarazel.de>
	<CA+Tgmoa2_8Ti9xez8wEH7Z_RJQv=cwPC=MeqUXcdVW=5-uMCig@mail.gmail.com>
	<20200402182346.6iffoadxu2hsbi2s@alap3.anarazel.de>
	<CA+TgmoboTL8cBt_P5Jx6RjL=Hk3Y8PbpGsh9KcH_MZuswVqdYA@mail.gmail.com>
	<CA+TgmoaGoQM74CMJhqg3_M7osWgkT2xSxo9aGVu9E4Nt+8Z4sg@mail.gmail.com>
	<20200402194750.7ze74b4t6b6o4cui@alap3.anarazel.de>
	<e3101c23-26b7-6fa2-14a8-af24c96602b5@pgmasters.net>
	<CA+TgmoaKhxPr0FWsEiZoXpMER44pMJ28L-Z7wcRrOKoGFwjiUQ@mail.gmail.com>



On 2020/04/04 4:22, Robert Haas wrote:
> On Thu, Apr 2, 2020 at 4:34 PM David Steele <david@pgmasters.net> wrote:
>> +1. These would be great tests to have and a win for pg_basebackup
>> overall but I don't think they should be a prerequisite for this commit.
> 
> Not to mention the server. I can't say that I have a lot of confidence
> that all of the server behavior in this area is well-understood and
> sane.
> 
> I've pushed all the patches.

When there is a backup_manifest in the database cluster, it's included in
the backup even when --no-manifest is specified. ISTM that this is problematic
because the backup_manifest is obviously not valid for the backup.
So, isn't it better to always exclude the *existing* backup_manifest in the
cluster from the backup, like backup_label/tablespace_map? Patch attached.

Also I found the typo in the document. Patch attached.

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

Attachments: download all (2 files, .tar.gz)

skip_backup_manifest.patch (text/plain, 1.3K) download diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c index 5d94b9c229..de77534fec 100644 --- a/src/backend/replication/basebackup.c +++ b/src/backend/replication/basebackup.c @@ -266,6 +266,13 @@ static const struct exclude_list_item excludeFiles[] = {BACKUP_LABEL_FILE, false}, {TABLESPACE_MAP, false}, + /* + * If there's a backup_manifest, it belongs to a backup that was used + * to start this server. It is *not* correct for this backup. Our + * backup_manifest is injected into the backup separately if users want it. + */ + {"backup_manifest", false}, + {"postmaster.pid", false}, {"postmaster.opts", false}, diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c index 9088f1f80f..daaefa751c 100644 --- a/src/bin/pg_rewind/filemap.c +++ b/src/bin/pg_rewind/filemap.c @@ -113,6 +113,13 @@ static const struct exclude_list_item excludeFiles[] = {"backup_label", false}, /* defined as BACKUP_LABEL_FILE */ {"tablespace_map", false}, /* defined as TABLESPACE_MAP */ + /* + * If there's a backup_manifest, it belongs to a backup that was used + * to start this server. It is *not* correct for this backup. Our + * backup_manifest is injected into the backup separately if users want it. + */ + {"backup_manifest", false}, + {"postmaster.pid", false}, {"postmaster.opts", false},
typo.patch (text/plain, 675B) download diff --git a/doc/src/sgml/ref/pg_validatebackup.sgml b/doc/src/sgml/ref/pg_validatebackup.sgml index 19888dc196..20d445efb8 100644 --- a/doc/src/sgml/ref/pg_validatebackup.sgml +++ b/doc/src/sgml/ref/pg_validatebackup.sgml @@ -132,7 +132,7 @@ PostgreSQL documentation <listitem> <para> Exit as soon as a problem with the backup is detected. If this option - is not specified, <literal>pg_basebackup</literal> will continue + is not specified, <literal>pg_validatebackup</literal> will continue checking the backup even after a problem has been detected, and will report all problems detected as errors. </para>

view thread (243+ messages)  latest in thread

Message-ID: <78f76a3d-1a28-a97d-0394-5c96985dd1c0@oss.nttdata.com>
Permalink:  ../78f76a3d-1a28-a97d-0394-5c96985dd1c0@oss.nttdata.com/
Also on:    postgresql.org/message-id/78f76a3d-1a28-a97d-0394-5c96985dd1c0@oss.nttdata.com

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: masao.fujii@oss.nttdata.com, robertmhaas@gmail.com, david@pgmasters.net, andres@anarazel.de, noah@leadboat.com, sfrost@snowman.net, amit.kapila16@gmail.com, suraj.kharage@enterprisedb.com, tushar.ahuja@enterprisedb.com, rajkumar.raghuwanshi@enterprisedb.com, rushabh.lathia@gmail.com, nospam-pg-abuse@bloodgate.com, andrew.dunstan@2ndquadrant.com, jeevan.chalke@enterprisedb.com, vignesh21@gmail.com
  Subject: Re: backup manifests
  In-Reply-To: <78f76a3d-1a28-a97d-0394-5c96985dd1c0@oss.nttdata.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox