Subject: Re: backup manifests
From: tushar <tushar.ahuja@enterprisedb.com>
To: Robert Haas <robertmhaas@gmail.com>,
 Suraj Kharage <suraj.kharage@enterprisedb.com>
Cc: Rushabh Lathia <rushabh.lathia@gmail.com>,
 Tels <nospam-pg-abuse@bloodgate.com>, David Steele <david@pgmasters.net>,
 Andrew Dunstan <andrew.dunstan@2ndquadrant.com>,
 PostgreSQL Hackers <pgsql-hackers@postgresql.org>,
 Jeevan Chalke <jeevan.chalke@enterprisedb.com>,
 vignesh C <vignesh21@gmail.com>
References: 
 <CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com>
 <CA+TgmoYTET9JosD9kS2QG28V=yvXGucq7dVNV9BPaZ=itGNPZw@mail.gmail.com>
 <CAF1DzPVuFa26xDsKG+Vbj+zLN8P4cXJ5Gdi0w8AUQ1NBKM3=HQ@mail.gmail.com>
 <CA+TgmoZn6K4KiBvcAfYOxUwv-t9oot0ZULdPbOG5k8prpEVVZA@mail.gmail.com>
 <CAF1DzPVeJdV3q4oiCBaTs5jMoUYkqNRicAYR-DAHuHGA7ccO=w@mail.gmail.com>
 <CAF1DzPXcCgXP2awVaH4uHovPo77JdLYPAYShzwnSPMVzdUmxSQ@mail.gmail.com>
 <CA+TgmoZdJ=vBoDU1Skv52Prybp36kMvmpTDBXbmoky0owNgMiA@mail.gmail.com>
 <CAF1DzPW1CAjR7FG86b-OL+hAP2xGqmBUtv0WtRhDz+r+Mwy-Hw@mail.gmail.com>
 <CA+TgmoYY5LXfbmeCO3gpCoCJcqNj0125PO7B7ecuJjnoEdB08A@mail.gmail.com>
 <CAF1DzPXOofDsdSqgEtzpegfcvqB_1TND_h_8uQrvJuVvx-4pDA@mail.gmail.com>
 <CA+TgmobnSHW3upkS_x6D=cM9A3jT9ZA0pcH0FJitBfkyW7sFJA@mail.gmail.com>
 <CAF1DzPUAuY5HqwzBoyFssuN9eRYmTkAKYLPEika1bOTvV2W1YA@mail.gmail.com>
 <CA+TgmoZRTBiPyvQEwV79PU1ePTtSEo2UeVncrkJMbn1sU1gnRA@mail.gmail.com>
 <d97505a0-7693-3edb-e52d-07c0c1e4b3a4@enterprisedb.com>
Message-ID: <8d81bcd9-fa06-a8a2-10e2-8368580e8e4f@enterprisedb.com>
Date: Tue, 3 Mar 2020 20:19:42 +0530
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <d97505a0-7693-3edb-e52d-07c0c1e4b3a4@enterprisedb.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Precedence: bulk

On 3/3/20 4:04 PM, tushar wrote:
> Thanks Robert.  After applying all the 5 patches (v8-00*) against PG 
> v13 (commit id -afb5465e0cfce7637066eaaaeecab30b0f23fbe3) , 

There is a scenario where pg_validatebackup is not throwing an error if 
some file deleted from pg_wal/ folder and  but later at the time of 
restoring - we are getting an error

[centos@tushar-ldap-docker bin]$ ./pg_basebackup  -D test1

[centos@tushar-ldap-docker bin]$ ls test1/pg_wal/
000000010000000000000010  archive_status

[centos@tushar-ldap-docker bin]$ rm -rf test1/pg_wal/*

[centos@tushar-ldap-docker bin]$ ./pg_validatebackup test1
pg_validatebackup: * manifest_checksum = 
88f1ed995c83e86252466a2c88b3e660a69cfc76c169991134b101c4f16c9df7
pg_validatebackup: backup successfully verified

[centos@tushar-ldap-docker bin]$ ./pg_ctl -D test1 start -o '-p 3333'
waiting for server to start....2020-03-02 20:05:22.732 IST [21441] LOG:  
starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by gcc 
(GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
2020-03-02 20:05:22.733 IST [21441] LOG:  listening on IPv6 address 
"::1", port 3333
2020-03-02 20:05:22.733 IST [21441] LOG:  listening on IPv4 address 
"127.0.0.1", port 3333
2020-03-02 20:05:22.736 IST [21441] LOG:  listening on Unix socket 
"/tmp/.s.PGSQL.3333"
2020-03-02 20:05:22.739 IST [21442] LOG:  database system was 
interrupted; last known up at 2020-03-02 20:04:35 IST
2020-03-02 20:05:22.739 IST [21442] LOG:  creating missing WAL directory 
"pg_wal/archive_status"
2020-03-02 20:05:22.886 IST [21442] LOG:  invalid checkpoint record
2020-03-02 20:05:22.886 IST [21442] FATAL:  could not locate required 
checkpoint record
2020-03-02 20:05:22.886 IST [21442] HINT:  If you are restoring from a 
backup, touch 
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/recovery.signal" and 
add required recovery options.
     If you are not restoring from a backup, try removing the file 
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label".
     Be careful: removing 
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label" will 
result in a corrupt cluster if restoring from a backup.
2020-03-02 20:05:22.886 IST [21441] LOG:  startup process (PID 21442) 
exited with exit code 1
2020-03-02 20:05:22.886 IST [21441] LOG:  aborting startup due to 
startup process failure
2020-03-02 20:05:22.889 IST [21441] LOG:  database system is shut down
  stopped waiting
pg_ctl: could not start server
Examine the log output.
[centos@tushar-ldap-docker bin]$

-- 
regards,tushar
EnterpriseDB  https://www.enterprisedb.com/
The Enterprise PostgreSQL Company