Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ci4TQ-0008Km-7y for pgsql-hackers@arkaria.postgresql.org; Sun, 26 Feb 2017 19:27:12 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1ci4TP-0007X2-R6 for pgsql-hackers@arkaria.postgresql.org; Sun, 26 Feb 2017 19:27:11 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1ci4TO-0007Wd-Qm for pgsql-hackers@postgresql.org; Sun, 26 Feb 2017 19:27:10 +0000 Received: from gauss.credativ.com ([93.94.130.89]) by makus.postgresql.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1ci4TL-0000Fc-S2 for pgsql-hackers@postgresql.org; Sun, 26 Feb 2017 19:27:09 +0000 Received: from gauss.credativ.com (localhost [127.0.0.1]) by gauss.credativ.com (Postfix) with ESMTP id C6E701E18C1; Sun, 26 Feb 2017 20:27:04 +0100 (CET) Received: from fock (ppp-88-217-106-253.dynamic.mnet-online.de [88.217.106.253]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: mba@gauss.credativ.com) by gauss.credativ.com (Postfix) with ESMTPSA id 614261E186E; Sun, 26 Feb 2017 20:27:04 +0100 (CET) Message-ID: <1488137223.24065.7.camel@credativ.de> Subject: Re: gitlab post-mortem: pg_basebackup waiting for checkpoint From: Michael Banck To: Robert Haas Cc: Alvaro Herrera , Magnus Hagander , Jim Nasby , Pg Hackers Date: Sun, 26 Feb 2017 20:27:03 +0100 In-Reply-To: References: <20170214210651.ldwzk6frxxclbomw@alvherre.pgsql> Organization: credativ GmbH Content-Type: multipart/mixed; boundary="=-+Zukf52EKjOQtBWcvG6o" X-Mailer: Evolution 3.12.9-1+b1 Mime-Version: 1.0 X-Pg-Spam-Score: -1.9 (-) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-hackers Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org --=-+Zukf52EKjOQtBWcvG6o Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit Hi, Am Dienstag, den 14.02.2017, 18:18 -0500 schrieb Robert Haas: > On Tue, Feb 14, 2017 at 4:06 PM, Alvaro Herrera > wrote: > > I'd rather have a --quiet mode instead. If you're running it by hand, > > you're likely to omit the switch, whereas when writing the cron job > > you're going to notice lack of switch even before you let the job run > > once. > > Well, that might've been a better way to design it, but changing it > now would break backward compatibility and I'm not really sure that's > a good idea. Even if it is, it's a separate concern from whether or > not in the less-quiet mode we should point out that we're waiting for > a checkpoint on the server side. ISTM the consensus is that there should be no output in regular mode, but a message should be displayed in verbose and progress mode. So I went forth and also added a message in progress mode (unless verbose messages are requested anyway). Regarding the documentation, I tried to clarify the difference between the checkpoint types a bit more, but I think any further action is probably a larger rewrite of the documentation on this topic. So attached are two patches, I've split it up in the documentation and the code output part. I'll add it as one commitfest entry in the "Clients" section though, as it's not really a big patch, unless somebody thinks it should have a secon entry in "Documentation"? Michael -- Michael Banck Projektleiter / Senior Berater Tel.: +49 2166 9901-171 Fax: +49 2166 9901-100 Email: michael.banck@credativ.de credativ GmbH, HRB Mönchengladbach 12080 USt-ID-Nummer: DE204566209 Trompeterallee 108, 41189 Mönchengladbach Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer --=-+Zukf52EKjOQtBWcvG6o Content-Disposition: attachment; filename*0=0001-Documentation-updates-regarding-checkpoints-for-base.pat; filename*1=ch Content-Transfer-Encoding: base64 Content-Type: text/x-patch; name="0001-Documentation-updates-regarding-checkpoints-for-base.patch"; charset="UTF-8" RnJvbSBiY2JlMTk4NTVmOWY5NGVhZGY5ZTQ3YTdmM2I5YTkyMGE3ZjJhNjE2 IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBNaWNoYWVsIEJhbmNr IDxtaWNoYWVsLmJhbmNrQGNyZWRhdGl2LmRlPgpEYXRlOiBTdW4sIDI2IEZl YiAyMDE3IDE4OjA2OjQwICswMTAwClN1YmplY3Q6IFtQQVRDSCAxLzJdIERv Y3VtZW50YXRpb24gdXBkYXRlcyByZWdhcmRpbmcgY2hlY2twb2ludHMgZm9y CiBiYXNlYmFja3Vwcy4KCk1lbnRpb24gdGhhdCBmYXN0IGFuZCBpbW1lZGlh dGUgY2hlY2twb2ludHMgYXJlIHRoZSBzYW1lLCBhbmQgYWRkIGEgcGFyYWdy YXBoIHRvCnRoZSBwZ19iYXNlYmFja3VwIGRvY3VtZW50YXRpb24gYWJvdXQg dGhlIGNoZWNrcG9pbnQgdGFrZW4gb24gdGhlIHJlbW90ZSBzZXJ2ZXIuCi0t LQogZG9jL3NyYy9zZ21sL2JhY2t1cC5zZ21sICAgICAgICAgICAgfCAgMyAr Ky0KIGRvYy9zcmMvc2dtbC9yZWYvcGdfYmFzZWJhY2t1cC5zZ21sIHwgMTAg KysrKysrKysrLQogMiBmaWxlcyBjaGFuZ2VkLCAxMSBpbnNlcnRpb25zKCsp LCAyIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2RvYy9zcmMvc2dtbC9i YWNrdXAuc2dtbCBiL2RvYy9zcmMvc2dtbC9iYWNrdXAuc2dtbAppbmRleCA1 ZjAwOWVlLi45NDg1ZDg3IDEwMDY0NAotLS0gYS9kb2Mvc3JjL3NnbWwvYmFj a3VwLnNnbWwKKysrIGIvZG9jL3NyYy9zZ21sL2JhY2t1cC5zZ21sCkBAIC04 NjIsNyArODYyLDggQEAgU0VMRUNUIHBnX3N0YXJ0X2JhY2t1cCgnbGFiZWwn LCBmYWxzZSwgZmFsc2UpOwogICAgICA8eHJlZiBsaW5rZW5kPSJndWMtY2hl Y2twb2ludC1jb21wbGV0aW9uLXRhcmdldCI+KS4gIFRoaXMgaXMKICAgICAg dXN1YWxseSB3aGF0IHlvdSB3YW50LCBiZWNhdXNlIGl0IG1pbmltaXplcyB0 aGUgaW1wYWN0IG9uIHF1ZXJ5CiAgICAgIHByb2Nlc3NpbmcuICBJZiB5b3Ug d2FudCB0byBzdGFydCB0aGUgYmFja3VwIGFzIHNvb24gYXMKLSAgICAgcG9z c2libGUsIGNoYW5nZSB0aGUgc2Vjb25kIHBhcmFtZXRlciB0byA8bGl0ZXJh bD50cnVlPC8+LgorICAgICBwb3NzaWJsZSwgY2hhbmdlIHRoZSBzZWNvbmQg cGFyYW1ldGVyIHRvIDxsaXRlcmFsPnRydWU8Lz4sIHdoaWNoIHdpbGwKKyAg ICAgaXNzdWUgYW4gaW1tZWRpYXRlIGNoZWNrcG9pbnQgdXNpbmcgYXMgbXVj aCBJL08gYXMgcG9zc2libGUuCiAgICAgPC9wYXJhPgogCiAgICAgPHBhcmE+ CmRpZmYgLS1naXQgYS9kb2Mvc3JjL3NnbWwvcmVmL3BnX2Jhc2ViYWNrdXAu c2dtbCBiL2RvYy9zcmMvc2dtbC9yZWYvcGdfYmFzZWJhY2t1cC5zZ21sCmlu ZGV4IGM5ZGQ2MmMuLmMxOTc2MzAgMTAwNjQ0Ci0tLSBhL2RvYy9zcmMvc2dt bC9yZWYvcGdfYmFzZWJhY2t1cC5zZ21sCisrKyBiL2RvYy9zcmMvc2dtbC9y ZWYvcGdfYmFzZWJhY2t1cC5zZ21sCkBAIC00MTksNyArNDE5LDcgQEAgUG9z dGdyZVNRTCBkb2N1bWVudGF0aW9uCiAgICAgICA8dGVybT48b3B0aW9uPi0t Y2hlY2twb2ludD08cmVwbGFjZWFibGUgY2xhc3M9InBhcmFtZXRlciI+ZmFz dHxzcHJlYWQ8L3JlcGxhY2VhYmxlPjwvb3B0aW9uPjwvdGVybT4KICAgICAg IDxsaXN0aXRlbT4KICAgICAgICA8cGFyYT4KLSAgICAgICAgU2V0cyBjaGVj a3BvaW50IG1vZGUgdG8gZmFzdCBvciBzcHJlYWQgKGRlZmF1bHQpIChzZWUg PHhyZWYgbGlua2VuZD0iYmFja3VwLWxvd2xldmVsLWJhc2UtYmFja3VwIj4p LgorICAgICAgICBTZXRzIGNoZWNrcG9pbnQgbW9kZSB0byBmYXN0IChpbW1l ZGlhdGUpIG9yIHNwcmVhZCAoZGVmYXVsdCkgKHNlZSA8eHJlZiBsaW5rZW5k PSJiYWNrdXAtbG93bGV2ZWwtYmFzZS1iYWNrdXAiPikuCiAgICAgICAgPC9w YXJhPgogICAgICAgPC9saXN0aXRlbT4KICAgICAgPC92YXJsaXN0ZW50cnk+ CkBAIC02NjAsNiArNjYwLDE0IEBAIFBvc3RncmVTUUwgZG9jdW1lbnRhdGlv bgogICA8dGl0bGU+Tm90ZXM8L3RpdGxlPgogCiAgIDxwYXJhPgorICAgQXQg dGhlIGJlZ2lubmluZyBvZiB0aGUgYmFja3VwLCBhIGNoZWNrcG9pbnQgbmVl ZHMgdG8gYmUgd3JpdHRlbiBvbiB0aGUKKyAgIHNlcnZlciB0aGUgYmFja3Vw IGlzIHRha2VuIGZyb20uICBFc3BlY2lhbGx5IGlmIHRoZSBvcHRpb24KKyAg IDxsaXRlcmFsPi0tY2hlY2twb2ludD1mYXN0PC9saXRlcmFsPiBpcyBub3Qg dXNlZCwgdGhpcyBjYW4gdGFrZSBzb21lIHRpbWUKKyAgIGR1cmluZyB3aGlj aCA8YXBwbGljYXRpb24+cGdfYmFzZWJhY2t1cDwvYXBwbGljYXRpb24+IHdp bGwgYmUgaWRsZSBvbiB0aGUKKyAgIHNlcnZlciBpdCBpcyBydW5uaW5nIG9u LgorICA8L3BhcmE+CisKKyAgPHBhcmE+CiAgICBUaGUgYmFja3VwIHdpbGwg aW5jbHVkZSBhbGwgZmlsZXMgaW4gdGhlIGRhdGEgZGlyZWN0b3J5IGFuZCB0 YWJsZXNwYWNlcywKICAgIGluY2x1ZGluZyB0aGUgY29uZmlndXJhdGlvbiBm aWxlcyBhbmQgYW55IGFkZGl0aW9uYWwgZmlsZXMgcGxhY2VkIGluIHRoZQog ICAgZGlyZWN0b3J5IGJ5IHRoaXJkIHBhcnRpZXMsIGV4Y2VwdCBjZXJ0YWlu IHRlbXBvcmFyeSBmaWxlcyBtYW5hZ2VkIGJ5Ci0tIAoyLjEuNAoK --=-+Zukf52EKjOQtBWcvG6o Content-Disposition: attachment; filename*0=0002-Mention-initial-checkpoint-in-pg_basebackup-for-verb.pat; filename*1=ch Content-Transfer-Encoding: base64 Content-Type: text/x-patch; name="0002-Mention-initial-checkpoint-in-pg_basebackup-for-verb.patch"; charset="UTF-8" RnJvbSAxZTQwNTFkZmY5NzEwMzgyYjZiNGY2MzM3M2EzMDRjNmNlNzBjNGFj IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBNaWNoYWVsIEJhbmNr IDxtaWNoYWVsLmJhbmNrQGNyZWRhdGl2LmRlPgpEYXRlOiBTdW4sIDI2IEZl YiAyMDE3IDIwOjIzOjIxICswMTAwClN1YmplY3Q6IFtQQVRDSCAyLzJdIE1l bnRpb24gaW5pdGlhbCBjaGVja3BvaW50IGluIHBnX2Jhc2ViYWNrdXAgZm9y CiB2ZXJib3NlL3Byb2dlc3Mgb3V0cHV0LgoKQmVmb3JlIHRoZSBhY3R1YWwg ZGF0YSBkaXJlY3RvcnkgY29udGVudHMgYXJlIHN0cmVhbWVkLCBhIGNoZWNr cG9pbnQgaXMKdGFrZW4gb24gdGhlIHJlbW90ZSBzZXJ2ZXIuIEVzcGVjaWFs bHkgaWYgbm8gZmFzdCBjaGVja3BvaW50IGlzCnJlcXVlc3RlZCwgdGhpcyBj YW4gdGFrZSBxdWl0ZSBhIHdoaWxlIGR1cmluZyB3aGljaCB0aGUgcGdfYmFz ZWJhY2t1cApjb21tYW5kIGFwcGFyZW50bHkgc2l0cyBpZGxlIGRvaW5nIG5v dGhpbmcuCgpUbyBhbGVydCB0aGUgdXNlciB0aGF0IHdvcmsgaXMgYmVpbmcg ZG9uZSBvbiB0aGUgcmVtb3RlIHNlcnZlciwgbWVudGlvbgp0aGUgY2hlY2tw b2ludCBpZiB2ZXJib3NlIG9yIHByb2dyZXNzIG91dHB1dCBoYXMgYmVlbiBy ZXF1ZXN0ZWQuICBBcwpwZ19iYXNlYmFja3VwIGRvZXMgbm90IG91dHB1dCBh bnl0aGluZyBkdXJpbmcgcmVndWxhciBvcGVyYXRpb24sIG5vCmFkZGl0aW9u YWwgb3V0cHV0IGlzIHByaW50ZWQgaW4gdGhpcyBjYXNlLgoKQWxzbyBoYXJt b25pemUgc29tZSBvdGhlciB2ZXJib3NlIG1lc3NhZ2VzIGluIHBhc3Npbmcu Ci0tLQogc3JjL2Jpbi9wZ19iYXNlYmFja3VwL3BnX2Jhc2ViYWNrdXAuYyB8 IDE5ICsrKysrKysrKysrKysrKy0tLS0KIDEgZmlsZSBjaGFuZ2VkLCAxNSBp bnNlcnRpb25zKCspLCA0IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3Ny Yy9iaW4vcGdfYmFzZWJhY2t1cC9wZ19iYXNlYmFja3VwLmMgYi9zcmMvYmlu L3BnX2Jhc2ViYWNrdXAvcGdfYmFzZWJhY2t1cC5jCmluZGV4IGJjOTk3ZGMu LjRiNzVlNzYgMTAwNjQ0Ci0tLSBhL3NyYy9iaW4vcGdfYmFzZWJhY2t1cC9w Z19iYXNlYmFja3VwLmMKKysrIGIvc3JjL2Jpbi9wZ19iYXNlYmFja3VwL3Bn X2Jhc2ViYWNrdXAuYwpAQCAtMTc1Myw2ICsxNzUzLDE0IEBAIEJhc2VCYWNr dXAodm9pZCkKIAlpZiAobWF4cmF0ZSA+IDApCiAJCW1heHJhdGVfY2xhdXNl ID0gcHNwcmludGYoIk1BWF9SQVRFICV1IiwgbWF4cmF0ZSk7CiAKKwlpZiAo dmVyYm9zZSkKKwkJZnByaW50ZihzdGRlcnIsCisJCQkJXygiJXM6IGluaXRp YXRpbmcgYmFzZSBiYWNrdXAsIHdhaXRpbmcgZm9yIGNoZWNrcG9pbnQgdG8g Y29tcGxldGVcbiIpLAorCQkJCXByb2duYW1lKTsKKworCWlmIChzaG93cHJv Z3Jlc3MgJiYgIXZlcmJvc2UpCisJCWZwcmludGYoc3RkZXJyLCAid2FpdGlu ZyBmb3IgY2hlY2twb2ludFxuIik7CisKIAliYXNlYmtwID0KIAkJcHNwcmlu dGYoIkJBU0VfQkFDS1VQIExBQkVMICclcycgJXMgJXMgJXMgJXMgJXMgJXMi LAogCQkJCSBlc2NhcGVkX2xhYmVsLApAQCAtMTc5MCw2ICsxNzk4LDkgQEAg QmFzZUJhY2t1cCh2b2lkKQogCiAJc3RybGNweSh4bG9nc3RhcnQsIFBRZ2V0 dmFsdWUocmVzLCAwLCAwKSwgc2l6ZW9mKHhsb2dzdGFydCkpOwogCisJaWYg KHZlcmJvc2UpCisJCWZwcmludGYoc3RkZXJyLCBfKCIlczogY2hlY2twb2lu dCBjb21wbGV0ZWRcbiIpLCBwcm9nbmFtZSk7CisKIAkvKgogCSAqIDkuMyBh bmQgbGF0ZXIgc2VuZHMgdGhlIFRMSSBvZiB0aGUgc3RhcnRpbmcgcG9pbnQu IFdpdGggb2xkZXIgc2VydmVycywKIAkgKiBhc3N1bWUgaXQncyB0aGUgc2Ft ZSBhcyB0aGUgbGF0ZXN0IHRpbWVsaW5lIHJlcG9ydGVkIGJ5CkBAIC0xODAz LDggKzE4MTQsOCBAQCBCYXNlQmFja3VwKHZvaWQpCiAJTWVtU2V0KHhsb2dl bmQsIDAsIHNpemVvZih4bG9nZW5kKSk7CiAKIAlpZiAodmVyYm9zZSAmJiBp bmNsdWRld2FsICE9IE5PX1dBTCkKLQkJZnByaW50ZihzdGRlcnIsIF8oInRy YW5zYWN0aW9uIGxvZyBzdGFydCBwb2ludDogJXMgb24gdGltZWxpbmUgJXVc biIpLAotCQkJCXhsb2dzdGFydCwgc3RhcnR0bGkpOworCQlmcHJpbnRmKHN0 ZGVyciwgXygiJXM6IHRyYW5zYWN0aW9uIGxvZyBzdGFydCBwb2ludDogJXMg b24gdGltZWxpbmUgJXVcbiIpLAorCQkJCXByb2duYW1lLCB4bG9nc3RhcnQs IHN0YXJ0dGxpKTsKIAogCS8qCiAJICogR2V0IHRoZSBoZWFkZXIKQEAgLTE5 MDYsNyArMTkxNyw3IEBAIEJhc2VCYWNrdXAodm9pZCkKIAl9CiAJc3RybGNw eSh4bG9nZW5kLCBQUWdldHZhbHVlKHJlcywgMCwgMCksIHNpemVvZih4bG9n ZW5kKSk7CiAJaWYgKHZlcmJvc2UgJiYgaW5jbHVkZXdhbCAhPSBOT19XQUwp Ci0JCWZwcmludGYoc3RkZXJyLCAidHJhbnNhY3Rpb24gbG9nIGVuZCBwb2lu dDogJXNcbiIsIHhsb2dlbmQpOworCQlmcHJpbnRmKHN0ZGVyciwgXygiJXM6 IHRyYW5zYWN0aW9uIGxvZyBlbmQgcG9pbnQ6ICVzXG4iKSwgcHJvZ25hbWUs IHhsb2dlbmQpOwogCVBRY2xlYXIocmVzKTsKIAogCXJlcyA9IFBRZ2V0UmVz dWx0KGNvbm4pOwpAQCAtMjA0Nyw3ICsyMDU4LDcgQEAgQmFzZUJhY2t1cCh2 b2lkKQogCX0KIAogCWlmICh2ZXJib3NlKQotCQlmcHJpbnRmKHN0ZGVyciwg IiVzOiBiYXNlIGJhY2t1cCBjb21wbGV0ZWRcbiIsIHByb2duYW1lKTsKKwkJ ZnByaW50ZihzdGRlcnIsIF8oIiVzOiBiYXNlIGJhY2t1cCBjb21wbGV0ZWRc biIpLCBwcm9nbmFtZSk7CiB9CiAKIAotLSAKMi4xLjQKCg== --=-+Zukf52EKjOQtBWcvG6o Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit MIME-Version: 1.0 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers --=-+Zukf52EKjOQtBWcvG6o--