Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from ) id 1jEVUh-0002wh-6c for pgsql-hackers@arkaria.postgresql.org; Wed, 18 Mar 2020 10:00:11 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1jEVUf-0006YK-Ma for pgsql-hackers@arkaria.postgresql.org; Wed, 18 Mar 2020 10:00:09 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1jEVUf-0006Ws-Cw for pgsql-hackers@lists.postgresql.org; Wed, 18 Mar 2020 10:00:09 +0000 Received: from oss.nttdata.com ([49.212.34.109]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jEVUU-00085D-NX for pgsql-hackers@postgresql.org; Wed, 18 Mar 2020 10:00:07 +0000 Received: from hnk.local (sp49-97-94-94.msc.spmode.ne.jp [49.97.94.94]) by oss.nttdata.com (Postfix) with ESMTPSA id 0B18660299; Wed, 18 Mar 2020 18:59:52 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.102.2 at oss.nttdata.com Subject: Re: RecoveryWalAll and RecoveryWalStream wait events To: Atsushi Torikoshi Cc: Kyotaro Horiguchi , PostgreSQL-development References: <124997ee-096a-5d09-d8da-2c7a57d0816e@oss.nttdata.com> <20200218.142016.2107516686265665330.horikyota.ntt@gmail.com> <2ea0c6f1-adcd-f2a2-fe5a-92376ad38432@oss.nttdata.com> From: Fujii Masao Message-ID: Date: Wed, 18 Mar 2020 18:59:51 +0900 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------9EBB774B9106619C978D30A3" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------9EBB774B9106619C978D30A3 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 2020/03/18 17:56, Atsushi Torikoshi wrote: > > > On Tue, Mar 17, 2020 at 11:55 AM Fujii Masao > wrote: > > >  >    Waiting when WAL data is not available from any kind of sources > >  >    (local, archive or stream) before trying again to retrieve WAL data, > > > > I think 'local' means pg_wal here, but the comment on > > WaitForWALToBecomeAvailable() says checking pg_wal in > > standby mode is 'not documented', so I'm a little bit worried > > that users may be confused. > > This logic seems to be documented in high-availability.sgml. > > > Thanks! I didn't notice it. > I think you mean the below sentence. > > >  The standby server will also attempt to restore any WAL found in the standby cluster's pg_wal directory. I meant the following part in the doc. --------------------- At startup, the standby begins by restoring all WAL available in the archive location, calling restore_command. Once it reaches the end of WAL available there and restore_command fails, it tries to restore any WAL available in the pg_wal directory. If that fails, and streaming replication has been configured, the standby tries to connect to the primary server and start streaming WAL from the last valid record found in archive or pg_wal. If that fails or streaming replication is not configured, or if the connection is later disconnected, the standby goes back to step 1 and tries to restore the file from the archive again. This loop of retries from the archive, pg_wal, and via streaming replication goes on until the server is stopped or failover is triggered by a trigger file. --------------------- > It seems the comment on WaitForWALToBecomeAvailable() > does not go along with the high-availability.sgml, do we need > modification on the comment on the function? No, I think for now. But you'd like to improve the docs? > But, anyway, you think that "pg_wal" should be used instead > > of "local" here? > > > I don't have special opinion here. > It might be better because high-availability.sgml does not use > "local" but "pg_wal" for the explanation,  but I also feel it's > obvious in this context. Ok, I changed that from "local" to "pg_wal" in the patch for the master. Attached is the updated version of the patch. If you're OK with this, I'd like to commit two patches that I posted in this thread. Regards, -- Fujii Masao NTT DATA CORPORATION Advanced Platform Technology Group Research and Development Headquarters --------------9EBB774B9106619C978D30A3 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="improve_recovery_wait_event_for_master_v2.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="improve_recovery_wait_event_for_master_v2.patch" ZGlmZiAtLWdpdCBhL2RvYy9zcmMvc2dtbC9tb25pdG9yaW5nLnNnbWwgYi9kb2Mvc3JjL3Nn bWwvbW9uaXRvcmluZy5zZ21sCmluZGV4IDc2MjY5ODc4MDguLjg5ODUzYTE2ZDggMTAwNjQ0 Ci0tLSBhL2RvYy9zcmMvc2dtbC9tb25pdG9yaW5nLnNnbWwKKysrIGIvZG9jL3NyYy9zZ21s L21vbml0b3Jpbmcuc2dtbApAQCAtMTI0NCw3ICsxMjQ0LDcgQEAgcG9zdGdyZXMgICAyNzA5 MyAgMC4wICAwLjAgIDMwMDk2ICAyNzUyID8gICAgICAgIFNzICAgMTE6MzQgICAwOjAwIHBv c3RncmVzOiBzZXIKICAgICAgICAgIDxlbnRyeT5XYWl0aW5nIHRvIGFjcXVpcmUgYSBwaW4g b24gYSBidWZmZXIuPC9lbnRyeT4KICAgICAgICAgPC9yb3c+CiAgICAgICAgIDxyb3c+Ci0g ICAgICAgICA8ZW50cnkgbW9yZXJvd3M9IjEzIj48bGl0ZXJhbD5BY3Rpdml0eTwvbGl0ZXJh bD48L2VudHJ5PgorICAgICAgICAgPGVudHJ5IG1vcmVyb3dzPSIxMiI+PGxpdGVyYWw+QWN0 aXZpdHk8L2xpdGVyYWw+PC9lbnRyeT4KICAgICAgICAgIDxlbnRyeT48bGl0ZXJhbD5BcmNo aXZlck1haW48L2xpdGVyYWw+PC9lbnRyeT4KICAgICAgICAgIDxlbnRyeT5XYWl0aW5nIGlu IG1haW4gbG9vcCBvZiB0aGUgYXJjaGl2ZXIgcHJvY2Vzcy48L2VudHJ5PgogICAgICAgICA8 L3Jvdz4KQEAgLTEyNzYsMTcgKzEyNzYsOSBAQCBwb3N0Z3JlcyAgIDI3MDkzICAwLjAgIDAu MCAgMzAwOTYgIDI3NTIgPyAgICAgICAgU3MgICAxMTozNCAgIDA6MDAgcG9zdGdyZXM6IHNl cgogICAgICAgICAgPGVudHJ5PjxsaXRlcmFsPlBnU3RhdE1haW48L2xpdGVyYWw+PC9lbnRy eT4KICAgICAgICAgIDxlbnRyeT5XYWl0aW5nIGluIG1haW4gbG9vcCBvZiB0aGUgc3RhdGlz dGljcyBjb2xsZWN0b3IgcHJvY2Vzcy48L2VudHJ5PgogICAgICAgICA8L3Jvdz4KLSAgICAg ICAgPHJvdz4KLSAgICAgICAgIDxlbnRyeT48bGl0ZXJhbD5SZWNvdmVyeVdhbEFsbDwvbGl0 ZXJhbD48L2VudHJ5PgotICAgICAgICAgPGVudHJ5PldhaXRpbmcgZm9yIFdBTCBmcm9tIGEg c3RyZWFtIGF0IHJlY292ZXJ5LjwvZW50cnk+Ci0gICAgICAgIDwvcm93PgogICAgICAgICA8 cm93PgogICAgICAgICAgPGVudHJ5PjxsaXRlcmFsPlJlY292ZXJ5V2FsU3RyZWFtPC9saXRl cmFsPjwvZW50cnk+Ci0gICAgICAgICA8ZW50cnk+Ci0gICAgICAgICAgV2FpdGluZyB3aGVu IFdBTCBkYXRhIGlzIG5vdCBhdmFpbGFibGUgZnJvbSBhbnkga2luZCBvZiBzb3VyY2VzCi0g ICAgICAgICAgKGxvY2FsLCBhcmNoaXZlIG9yIHN0cmVhbSkgYmVmb3JlIHRyeWluZyBhZ2Fp biB0byByZXRyaWV2ZSBXQUwgZGF0YSwKLSAgICAgICAgICBhdCByZWNvdmVyeS4KLSAgICAg ICAgIDwvZW50cnk+CisgICAgICAgICA8ZW50cnk+V2FpdGluZyBmb3IgV0FMIGZyb20gYSBz dHJlYW0gYXQgcmVjb3ZlcnkuPC9lbnRyeT4KICAgICAgICAgPC9yb3c+CiAgICAgICAgIDxy b3c+CiAgICAgICAgICA8ZW50cnk+PGxpdGVyYWw+U3lzTG9nZ2VyTWFpbjwvbGl0ZXJhbD48 L2VudHJ5PgpAQCAtMTQ5Niw3ICsxNDg4LDcgQEAgcG9zdGdyZXMgICAyNzA5MyAgMC4wICAw LjAgIDMwMDk2ICAyNzUyID8gICAgICAgIFNzICAgMTE6MzQgICAwOjAwIHBvc3RncmVzOiBz ZXIKICAgICAgICAgIDxlbnRyeT5XYWl0aW5nIGZvciBjb25maXJtYXRpb24gZnJvbSByZW1v dGUgc2VydmVyIGR1cmluZyBzeW5jaHJvbm91cyByZXBsaWNhdGlvbi48L2VudHJ5PgogICAg ICAgICA8L3Jvdz4KICAgICAgICAgPHJvdz4KLSAgICAgICAgIDxlbnRyeSBtb3Jlcm93cz0i MiI+PGxpdGVyYWw+VGltZW91dDwvbGl0ZXJhbD48L2VudHJ5PgorICAgICAgICAgPGVudHJ5 IG1vcmVyb3dzPSIzIj48bGl0ZXJhbD5UaW1lb3V0PC9saXRlcmFsPjwvZW50cnk+CiAgICAg ICAgICA8ZW50cnk+PGxpdGVyYWw+QmFzZUJhY2t1cFRocm90dGxlPC9saXRlcmFsPjwvZW50 cnk+CiAgICAgICAgICA8ZW50cnk+V2FpdGluZyBkdXJpbmcgYmFzZSBiYWNrdXAgd2hlbiB0 aHJvdHRsaW5nIGFjdGl2aXR5LjwvZW50cnk+CiAgICAgICAgIDwvcm93PgpAQCAtMTUwOCw2 ICsxNTAwLDE0IEBAIHBvc3RncmVzICAgMjcwOTMgIDAuMCAgMC4wICAzMDA5NiAgMjc1MiA/ ICAgICAgICBTcyAgIDExOjM0ICAgMDowMCBwb3N0Z3Jlczogc2VyCiAgICAgICAgICA8ZW50 cnk+PGxpdGVyYWw+UmVjb3ZlcnlBcHBseURlbGF5PC9saXRlcmFsPjwvZW50cnk+CiAgICAg ICAgICA8ZW50cnk+V2FpdGluZyB0byBhcHBseSBXQUwgYXQgcmVjb3ZlcnkgYmVjYXVzZSBp dCBpcyBkZWxheWVkLjwvZW50cnk+CiAgICAgICAgIDwvcm93PgorICAgICAgICA8cm93Pgor ICAgICAgICAgPGVudHJ5PjxsaXRlcmFsPlJlY292ZXJ5UmV0cmlldmVSZXRyeUludGVydmFs PC9saXRlcmFsPjwvZW50cnk+CisgICAgICAgICA8ZW50cnk+CisgICAgICAgICAgV2FpdGlu ZyB3aGVuIFdBTCBkYXRhIGlzIG5vdCBhdmFpbGFibGUgZnJvbSBhbnkga2luZCBvZiBzb3Vy Y2VzCisgICAgICAgICAgKDxmaWxlbmFtZT5wZ193YWw8L2ZpbGVuYW1lPiwgYXJjaGl2ZSBv ciBzdHJlYW0pIGJlZm9yZSB0cnlpbmcKKyAgICAgICAgICBhZ2FpbiB0byByZXRyaWV2ZSBX QUwgZGF0YSwgYXQgcmVjb3ZlcnkuCisgICAgICAgICA8L2VudHJ5PgorICAgICAgICA8L3Jv dz4KICAgICAgICAgPHJvdz4KICAgICAgICAgIDxlbnRyeSBtb3Jlcm93cz0iNjgiPjxsaXRl cmFsPklPPC9saXRlcmFsPjwvZW50cnk+CiAgICAgICAgICA8ZW50cnk+PGxpdGVyYWw+QnVm RmlsZVJlYWQ8L2xpdGVyYWw+PC9lbnRyeT4KZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL2Fj Y2Vzcy90cmFuc2FtL3hsb2cuYyBiL3NyYy9iYWNrZW5kL2FjY2Vzcy90cmFuc2FtL3hsb2cu YwppbmRleCBkZTJkNGVlNTgyLi43OTNjMDc2ZGE2IDEwMDY0NAotLS0gYS9zcmMvYmFja2Vu ZC9hY2Nlc3MvdHJhbnNhbS94bG9nLmMKKysrIGIvc3JjL2JhY2tlbmQvYWNjZXNzL3RyYW5z YW0veGxvZy5jCkBAIC0xMjAzMSw3ICsxMjAzMSw3IEBAIFdhaXRGb3JXQUxUb0JlY29tZUF2 YWlsYWJsZShYTG9nUmVjUHRyIFJlY1B0ciwgYm9vbCByYW5kQWNjZXNzLAogCQkJCQkJCQkJ CSBXTF9MQVRDSF9TRVQgfCBXTF9USU1FT1VUIHwKIAkJCQkJCQkJCQkgV0xfRVhJVF9PTl9Q TV9ERUFUSCwKIAkJCQkJCQkJCQkgd2FpdF90aW1lLAotCQkJCQkJCQkJCSBXQUlUX0VWRU5U X1JFQ09WRVJZX1dBTF9TVFJFQU0pOworCQkJCQkJCQkJCSBXQUlUX0VWRU5UX1JFQ09WRVJZ X1JFVFJJRVZFX1JFVFJZX0lOVEVSVkFMKTsKIAkJCQkJCVJlc2V0TGF0Y2goJlhMb2dDdGwt PnJlY292ZXJ5V2FrZXVwTGF0Y2gpOwogCQkJCQkJbm93ID0gR2V0Q3VycmVudFRpbWVzdGFt cCgpOwogCQkJCQl9CkBAIC0xMjIyMSw3ICsxMjIyMSw3IEBAIFdhaXRGb3JXQUxUb0JlY29t ZUF2YWlsYWJsZShYTG9nUmVjUHRyIFJlY1B0ciwgYm9vbCByYW5kQWNjZXNzLAogCQkJCQko dm9pZCkgV2FpdExhdGNoKCZYTG9nQ3RsLT5yZWNvdmVyeVdha2V1cExhdGNoLAogCQkJCQkJ CQkJIFdMX0xBVENIX1NFVCB8IFdMX1RJTUVPVVQgfAogCQkJCQkJCQkJIFdMX0VYSVRfT05f UE1fREVBVEgsCi0JCQkJCQkJCQkgNTAwMEwsIFdBSVRfRVZFTlRfUkVDT1ZFUllfV0FMX0FM TCk7CisJCQkJCQkJCQkgNTAwMEwsIFdBSVRfRVZFTlRfUkVDT1ZFUllfV0FMX1NUUkVBTSk7 CiAJCQkJCVJlc2V0TGF0Y2goJlhMb2dDdGwtPnJlY292ZXJ5V2FrZXVwTGF0Y2gpOwogCQkJ CQlicmVhazsKIAkJCQl9CmRpZmYgLS1naXQgYS9zcmMvYmFja2VuZC9wb3N0bWFzdGVyL3Bn c3RhdC5jIGIvc3JjL2JhY2tlbmQvcG9zdG1hc3Rlci9wZ3N0YXQuYwppbmRleCBmOTI4N2I3 OTQyLi5kMjljMjExYTc2IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9wb3N0bWFzdGVyL3Bn c3RhdC5jCisrKyBiL3NyYy9iYWNrZW5kL3Bvc3RtYXN0ZXIvcGdzdGF0LmMKQEAgLTM2MDIs OSArMzYwMiw2IEBAIHBnc3RhdF9nZXRfd2FpdF9hY3Rpdml0eShXYWl0RXZlbnRBY3Rpdml0 eSB3KQogCQljYXNlIFdBSVRfRVZFTlRfUEdTVEFUX01BSU46CiAJCQlldmVudF9uYW1lID0g IlBnU3RhdE1haW4iOwogCQkJYnJlYWs7Ci0JCWNhc2UgV0FJVF9FVkVOVF9SRUNPVkVSWV9X QUxfQUxMOgotCQkJZXZlbnRfbmFtZSA9ICJSZWNvdmVyeVdhbEFsbCI7Ci0JCQlicmVhazsK IAkJY2FzZSBXQUlUX0VWRU5UX1JFQ09WRVJZX1dBTF9TVFJFQU06CiAJCQlldmVudF9uYW1l ID0gIlJlY292ZXJ5V2FsU3RyZWFtIjsKIAkJCWJyZWFrOwpAQCAtMzgyNCw2ICszODIxLDkg QEAgcGdzdGF0X2dldF93YWl0X3RpbWVvdXQoV2FpdEV2ZW50VGltZW91dCB3KQogCQljYXNl IFdBSVRfRVZFTlRfUkVDT1ZFUllfQVBQTFlfREVMQVk6CiAJCQlldmVudF9uYW1lID0gIlJl Y292ZXJ5QXBwbHlEZWxheSI7CiAJCQlicmVhazsKKwkJY2FzZSBXQUlUX0VWRU5UX1JFQ09W RVJZX1JFVFJJRVZFX1JFVFJZX0lOVEVSVkFMOgorCQkJZXZlbnRfbmFtZSA9ICJSZWNvdmVy eVJldHJpZXZlUmV0cnlJbnRlcnZhbCI7CisJCQlicmVhazsKIAkJCS8qIG5vIGRlZmF1bHQg Y2FzZSwgc28gdGhhdCBjb21waWxlciB3aWxsIHdhcm4gKi8KIAl9CiAKZGlmZiAtLWdpdCBh L3NyYy9pbmNsdWRlL3Bnc3RhdC5oIGIvc3JjL2luY2x1ZGUvcGdzdGF0LmgKaW5kZXggMWEx OTkyMWY4MC4uODUxZDBhNzI0NiAxMDA2NDQKLS0tIGEvc3JjL2luY2x1ZGUvcGdzdGF0LmgK KysrIGIvc3JjL2luY2x1ZGUvcGdzdGF0LmgKQEAgLTc2MSw3ICs3NjEsNiBAQCB0eXBlZGVm IGVudW0KIAlXQUlUX0VWRU5UX0xPR0lDQUxfQVBQTFlfTUFJTiwKIAlXQUlUX0VWRU5UX0xP R0lDQUxfTEFVTkNIRVJfTUFJTiwKIAlXQUlUX0VWRU5UX1BHU1RBVF9NQUlOLAotCVdBSVRf RVZFTlRfUkVDT1ZFUllfV0FMX0FMTCwKIAlXQUlUX0VWRU5UX1JFQ09WRVJZX1dBTF9TVFJF QU0sCiAJV0FJVF9FVkVOVF9TWVNMT0dHRVJfTUFJTiwKIAlXQUlUX0VWRU5UX1dBTF9SRUNF SVZFUl9NQUlOLApAQCAtODQ4LDcgKzg0Nyw4IEBAIHR5cGVkZWYgZW51bQogewogCVdBSVRf RVZFTlRfQkFTRV9CQUNLVVBfVEhST1RUTEUgPSBQR19XQUlUX1RJTUVPVVQsCiAJV0FJVF9F VkVOVF9QR19TTEVFUCwKLQlXQUlUX0VWRU5UX1JFQ09WRVJZX0FQUExZX0RFTEFZCisJV0FJ VF9FVkVOVF9SRUNPVkVSWV9BUFBMWV9ERUxBWSwKKwlXQUlUX0VWRU5UX1JFQ09WRVJZX1JF VFJJRVZFX1JFVFJZX0lOVEVSVkFMCiB9IFdhaXRFdmVudFRpbWVvdXQ7CiAKIC8qIC0tLS0t LS0tLS0K --------------9EBB774B9106619C978D30A3--