Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jNoXk-0000W5-5O for pgsql-hackers@arkaria.postgresql.org; Mon, 13 Apr 2020 02:09:48 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jNoXi-0005SG-01 for pgsql-hackers@arkaria.postgresql.org; Mon, 13 Apr 2020 02:09:46 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jNoXh-0005S9-Nr for pgsql-hackers@lists.postgresql.org; Mon, 13 Apr 2020 02:09:45 +0000 Received: from oss.nttdata.com ([49.212.34.109]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jNoXd-0007wE-Q7 for pgsql-hackers@postgresql.org; Mon, 13 Apr 2020 02:09:45 +0000 Received: from hnk.local (p2421237-ipbf2725funabasi.chiba.ocn.ne.jp [122.24.33.237]) by oss.nttdata.com (Postfix) with ESMTPSA id E87E9605CC; Mon, 13 Apr 2020 11:09:34 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.102.2 at oss.nttdata.com Subject: Re: backup manifests From: Fujii Masao To: Robert Haas Cc: David Steele , Andres Freund , Noah Misch , Stephen Frost , Amit Kapila , Suraj Kharage , tushar , Rajkumar Raghuwanshi , Rushabh Lathia , Tels , Andrew Dunstan , PostgreSQL Hackers , Jeevan Chalke , vignesh C References: <20200330185944.42bxysvem6b757ew@alap3.anarazel.de> <20200331225034.odt3l2w4h3dr6ggk@alap3.anarazel.de> <20200402172318.3kvfrrewxmemzpia@alap3.anarazel.de> <20200402182346.6iffoadxu2hsbi2s@alap3.anarazel.de> <20200402194750.7ze74b4t6b6o4cui@alap3.anarazel.de> <78f76a3d-1a28-a97d-0394-5c96985dd1c0@oss.nttdata.com> <777d236e-f46c-8099-bb5e-a6e4efc43999@oss.nttdata.com> Message-ID: <01e3ed3a-8729-5aaa-ca84-e60e3ca59db8@oss.nttdata.com> Date: Mon, 13 Apr 2020 11:09:34 +0900 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <777d236e-f46c-8099-bb5e-a6e4efc43999@oss.nttdata.com> Content-Type: multipart/mixed; boundary="------------411E658470ACA56EE4418D9A" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------411E658470ACA56EE4418D9A Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 2020/04/09 23:06, Fujii Masao wrote: > > > On 2020/04/09 2:35, Robert Haas wrote: >> On Wed, Apr 8, 2020 at 1:15 AM Fujii Masao wrote: >>> When there is a backup_manifest in the database cluster, it's included in >>> the backup even when --no-manifest is specified. ISTM that this is problematic >>> because the backup_manifest is obviously not valid for the backup. >>> So, isn't it better to always exclude the *existing* backup_manifest in the >>> cluster from the backup, like backup_label/tablespace_map? Patch attached. >>> >>> Also I found the typo in the document. Patch attached. >> >> Both patches look good. The second one is definitely a mistake on my >> part, and the first one seems like a totally reasonable change. >> Thanks! > > Thanks for reviewing them! I pushed them. I found other minor issues. + When this option is specified with a value of yes + or force-escape, a backup manifest is created force-escape should be force-encode. Patch attached. - while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP", + while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:", "m:" seems unnecessary, so should be removed? Patch attached. + if (strcmp(basedir, "-") == 0) + { + char header[512]; + PQExpBufferData buf; + + initPQExpBuffer(&buf); + ReceiveBackupManifestInMemory(conn, &buf); backup_manifest should be received only when the manifest is enabled, so ISTM that the flag "manifest" should be checked in the above if-condition. Thought? Patch attached. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION --------------411E658470ACA56EE4418D9A Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="fix_typo_in_protocol_sgml.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="fix_typo_in_protocol_sgml.patch" ZGlmZiAtLWdpdCBhL2RvYy9zcmMvc2dtbC9wcm90b2NvbC5zZ21sIGIvZG9jL3NyYy9zZ21s L3Byb3RvY29sLnNnbWwKaW5kZXggNTM2ZGU5YTY5OC4uODcyOTI3MzliNSAxMDA2NDQKLS0t IGEvZG9jL3NyYy9zZ21sL3Byb3RvY29sLnNnbWwKKysrIGIvZG9jL3NyYy9zZ21sL3Byb3Rv Y29sLnNnbWwKQEAgLTI1NzgsMTkgKzI1NzgsMTkgQEAgVGhlIGNvbW1hbmRzIGFjY2VwdGVk IGluIHJlcGxpY2F0aW9uIG1vZGUgYXJlOgogICAgICAgIDwvdmFybGlzdGVudHJ5PgogCiAg ICAgICAgPHZhcmxpc3RlbnRyeT4KLSAgICAgICAgPHRlcm0+PGxpdGVyYWw+TUFOSUZFU1Q8 L2xpdGVyYWw+PC90ZXJtPgorICAgICAgICA8dGVybT48bGl0ZXJhbD5NQU5JRkVTVDwvbGl0 ZXJhbD4gPHJlcGxhY2VhYmxlPm1hbmlmZXN0X29wdGlvbjwvcmVwbGFjZWFibGU+PC90ZXJt PgogICAgICAgICA8bGlzdGl0ZW0+CiAgICAgICAgICA8cGFyYT4KICAgICAgICAgICBXaGVu IHRoaXMgb3B0aW9uIGlzIHNwZWNpZmllZCB3aXRoIGEgdmFsdWUgb2YgPGxpdGVyYWw+eWVz PC9saXRlcmFsPgotICAgICAgICAgIG9yIDxsaXRlcmFsPmZvcmNlLWVzY2FwZTwvbGl0ZXJh bD4sIGEgYmFja3VwIG1hbmlmZXN0IGlzIGNyZWF0ZWQKKyAgICAgICAgICBvciA8bGl0ZXJh bD5mb3JjZS1lbmNvZGU8L2xpdGVyYWw+LCBhIGJhY2t1cCBtYW5pZmVzdCBpcyBjcmVhdGVk CiAgICAgICAgICAgYW5kIHNlbnQgYWxvbmcgd2l0aCB0aGUgYmFja3VwLiAgVGhlIG1hbmlm ZXN0IGlzIGEgbGlzdCBvZiBldmVyeQogICAgICAgICAgIGZpbGUgcHJlc2VudCBpbiB0aGUg YmFja3VwIHdpdGggdGhlIGV4Y2VwdGlvbiBvZiBhbnkgV0FMIGZpbGVzIHRoYXQKICAgICAg ICAgICBtYXkgYmUgaW5jbHVkZWQuIEl0IGFsc28gc3RvcmVzIHRoZSBzaXplLCBsYXN0IG1v ZGlmaWNhdGlvbiB0aW1lLCBhbmQKICAgICAgICAgICBhbiBvcHRpb25hbCBjaGVja3N1bSBm b3IgZWFjaCBmaWxlLgotICAgICAgICAgIEEgdmFsdWUgb2YgPGxpdGVyYWw+Zm9yY2UtZXNj YXBlPC9saXRlcmFsPiBmb3JjZXMgYWxsIGZpbGVuYW1lcworICAgICAgICAgIEEgdmFsdWUg b2YgPGxpdGVyYWw+Zm9yY2UtZW5jb2RlPC9saXRlcmFsPiBmb3JjZXMgYWxsIGZpbGVuYW1l cwogICAgICAgICAgIHRvIGJlIGhleC1lbmNvZGVkOyBvdGhlcndpc2UsIHRoaXMgdHlwZSBv ZiBlbmNvZGluZyBpcyBwZXJmb3JtZWQgb25seQogICAgICAgICAgIGZvciBmaWxlcyB3aG9z ZSBuYW1lcyBhcmUgbm9uLVVURjggb2N0ZXQgc2VxdWVuY2VzLgotICAgICAgICAgIDxsaXRl cmFsPmZvcmNlLWVzY2FwZTwvbGl0ZXJhbD4gaXMgaW50ZW5kZWQgcHJpbWFyaWx5IGZvciB0 ZXN0aW5nCisgICAgICAgICAgPGxpdGVyYWw+Zm9yY2UtZW5jb2RlPC9saXRlcmFsPiBpcyBp bnRlbmRlZCBwcmltYXJpbHkgZm9yIHRlc3RpbmcKICAgICAgICAgICBwdXJwb3NlcywgdG8g YmUgc3VyZSB0aGF0IGNsaWVudHMgd2hpY2ggcmVhZCB0aGUgYmFja3VwIG1hbmlmZXN0CiAg ICAgICAgICAgY2FuIGhhbmRsZSB0aGlzIGNhc2UuIEZvciBjb21wYXRpYmlsaXR5IHdpdGgg cHJldmlvdXMgcmVsZWFzZXMsCiAgICAgICAgICAgdGhlIGRlZmF1bHQgaXMgPGxpdGVyYWw+ TUFOSUZFU1QgJ25vJzwvbGl0ZXJhbD4uCkBAIC0yNTk5LDcgKzI1OTksNyBAQCBUaGUgY29t bWFuZHMgYWNjZXB0ZWQgaW4gcmVwbGljYXRpb24gbW9kZSBhcmU6CiAgICAgICAgPC92YXJs aXN0ZW50cnk+CiAKICAgICAgICA8dmFybGlzdGVudHJ5PgotICAgICAgICA8dGVybT48bGl0 ZXJhbD5NQU5JRkVTVF9DSEVDS1NVTVM8L2xpdGVyYWw+PC90ZXJtPgorICAgICAgICA8dGVy bT48bGl0ZXJhbD5NQU5JRkVTVF9DSEVDS1NVTVM8L2xpdGVyYWw+IDxyZXBsYWNlYWJsZT5j aGVja3N1bV9hbGdvcml0aG08L3JlcGxhY2VhYmxlPjwvdGVybT4KICAgICAgICAgPGxpc3Rp dGVtPgogICAgICAgICAgPHBhcmE+CiAgICAgICAgICAgU3BlY2lmaWVzIHRoZSBhbGdvcml0 aG0gdGhhdCBzaG91bGQgYmUgYXBwbGllZCB0byBlYWNoIGZpbGUgaW5jbHVkZWQK --------------411E658470ACA56EE4418D9A Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="remove_unnecessary_getopt_option.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="remove_unnecessary_getopt_option.patch" ZGlmZiAtLWdpdCBhL3NyYy9iaW4vcGdfYmFzZWJhY2t1cC9wZ19iYXNlYmFja3VwLmMgYi9z cmMvYmluL3BnX2Jhc2ViYWNrdXAvcGdfYmFzZWJhY2t1cC5jCmluZGV4IGRlMDk4YjM1NTgu LmYxYWY4ZjkwNGEgMTAwNjQ0Ci0tLSBhL3NyYy9iaW4vcGdfYmFzZWJhY2t1cC9wZ19iYXNl YmFja3VwLmMKKysrIGIvc3JjL2Jpbi9wZ19iYXNlYmFja3VwL3BnX2Jhc2ViYWNrdXAuYwpA QCAtMjI3MSw3ICsyMjcxLDcgQEAgbWFpbihpbnQgYXJnYywgY2hhciAqKmFyZ3YpCiAKIAlh dGV4aXQoY2xlYW51cF9kaXJlY3Rvcmllc19hdGV4aXQpOwogCi0Jd2hpbGUgKChjID0gZ2V0 b3B0X2xvbmcoYXJnYywgYXJndiwgIkNEOkY6cjpSUzpUOlg6bDpuTnpaOmQ6YzpoOnA6VTpz OndXa3ZQbToiLAorCXdoaWxlICgoYyA9IGdldG9wdF9sb25nKGFyZ2MsIGFyZ3YsICJDRDpG OnI6UlM6VDpYOmw6bk56WjpkOmM6aDpwOlU6czp3V2t2UCIsCiAJCQkJCQkJbG9uZ19vcHRp b25zLCAmb3B0aW9uX2luZGV4KSkgIT0gLTEpCiAJewogCQlzd2l0Y2ggKGMpCg== --------------411E658470ACA56EE4418D9A Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="receive_manifest_only_when_enabled.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="receive_manifest_only_when_enabled.patch" ZGlmZiAtLWdpdCBhL3NyYy9iaW4vcGdfYmFzZWJhY2t1cC9wZ19iYXNlYmFja3VwLmMgYi9z cmMvYmluL3BnX2Jhc2ViYWNrdXAvcGdfYmFzZWJhY2t1cC5jCmluZGV4IGYxYWY4ZjkwNGEu LjY1Y2ExYjE2ZjAgMTAwNjQ0Ci0tLSBhL3NyYy9iaW4vcGdfYmFzZWJhY2t1cC9wZ19iYXNl YmFja3VwLmMKKysrIGIvc3JjL2Jpbi9wZ19iYXNlYmFja3VwL3BnX2Jhc2ViYWNrdXAuYwpA QCAtMTIxMSw3ICsxMjExLDcgQEAgUmVjZWl2ZVRhckZpbGUoUEdjb25uICpjb25uLCBQR3Jl c3VsdCAqcmVzLCBpbnQgcm93bnVtKQogCSAqIHdlJ3JlIHdyaXRpbmcgYSB0YXJmaWxlIHRv IHN0ZG91dCwgd2UgZG9uJ3QgaGF2ZSB0aGF0IG9wdGlvbiwgc28KIAkgKiBpbmNsdWRlIGl0 IGluIHRoZSBvbmUgdGFyZmlsZSB3ZSd2ZSBnb3QuCiAJICovCi0JaWYgKHN0cmNtcChiYXNl ZGlyLCAiLSIpID09IDApCisJaWYgKHN0cmNtcChiYXNlZGlyLCAiLSIpID09IDAgJiYgbWFu aWZlc3QpCiAJewogCQljaGFyCQloZWFkZXJbNTEyXTsKIAkJUFFFeHBCdWZmZXJEYXRhCWJ1 ZjsK --------------411E658470ACA56EE4418D9A--