Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rkUyH-00D9aE-33 for pgsql-hackers@arkaria.postgresql.org; Wed, 13 Mar 2024 20:13:05 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rkUyE-00Cibm-DD for pgsql-hackers@arkaria.postgresql.org; Wed, 13 Mar 2024 20:13:02 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rkUyE-00Cibe-2l for pgsql-hackers@lists.postgresql.org; Wed, 13 Mar 2024 20:13:02 +0000 Received: from mail.thelabyrinth.net ([45.56.70.56]) by magus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rkUyB-004MbA-Nr for pgsql-hackers@lists.postgresql.org; Wed, 13 Mar 2024 20:13:01 +0000 Received: from [10.5.0.2] (unknown [31.222.254.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: dsteele) by mail.thelabyrinth.net (Postfix) with ESMTPSA id 13CD754666; Wed, 13 Mar 2024 20:12:56 +0000 (UTC) Content-Type: multipart/mixed; boundary="------------hMJSX9YBRgQL63k90JTy0Qrw" Message-ID: Date: Thu, 14 Mar 2024 09:12:52 +1300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Add basic tests for the low-level backup method. To: Michael Paquier Cc: pgsql-hackers@lists.postgresql.org References: <03e4cf19-fb76-4891-accc-edd429a25b21@pgmasters.net> <431ecc5d-8c07-4ee1-b05e-3ca536e606bb@pgmasters.net> <4980aba2-9cd4-4bd6-b2e1-e873639ab7ee@pgmasters.net> Content-Language: en-US From: David Steele In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------hMJSX9YBRgQL63k90JTy0Qrw Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 3/13/24 19:15, Michael Paquier wrote: > On Wed, Mar 13, 2024 at 01:12:28PM +1300, David Steele wrote: >> >> Not sure what to look for here. There are no distinct messages for crash >> recovery. Perhaps there should be? > > The closest thing I can think of here would be "database system was > not properly shut down; automatic recovery in progress" as we don't > have InArchiveRecovery, after checking that the canary is missing. If > you don't like this suggestion, feel free to say so, of course :) That works for me. I think I got it confused with "database system was interrupted..." when I was looking at the success vs. fail logs. >> Sure, I added a check for the new log message when recovering with a >> backup_label. > > +ok($node_replica->log_contains('completed backup recovery with redo LSN'), > + 'verify backup recovery performed with backup_label'); > > Okay for this choice. I was thinking first about "starting backup > recovery with redo LSN", closer to the area where the backup_label is > read. I think you are right that the start message is better since it can only appear once when the backup_label is found. The completed message could in theory appear after a restart, though the backup_label must have been found at some point. Regards, -David --------------hMJSX9YBRgQL63k90JTy0Qrw Content-Type: text/plain; charset=UTF-8; name="low-level-backup-test-v3.patch" Content-Disposition: attachment; filename="low-level-backup-test-v3.patch" Content-Transfer-Encoding: base64 RnJvbSA5MWY4ZThhMzkwNzEzNzYwMTE2OTkwYmUwNWQ5NmY4YTdlMGQxYmRlIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBEYXZpZCBTdGVlbGUgPGRhdmlkQHBnbWFzdGVycy5u ZXQ+CkRhdGU6IFdlZCwgMTMgTWFyIDIwMjQgMjA6MDI6NDAgKzAwMDAKU3ViamVjdDogQWRk IGJhc2ljIHRlc3RzIGZvciB0aGUgbG93LWxldmVsIGJhY2t1cCBtZXRob2QuCgpUaGVyZSBh cmUgY3VycmVudGx5IG5vIHRlc3RzIGZvciB0aGUgbG93LWxldmVsIGJhY2t1cCBtZXRob2Qu IHBnX2JhY2t1cF9zdGFydCgpCmFuZCBwZ19iYWNrdXBfc3RvcCgpIGFyZSBjYWxsZWQgYnV0 IG5vdCBleGVyY2lzZWQgaW4gYW55IHJlYWwgZmFzaGlvbi4KClRoZXJlIGlzIGEgbG90IG1v cmUgdGhhdCBjYW4gYmUgZG9uZSwgYnV0IHRoaXMgYXQgbGVhc3QgcHJvdmlkZXMgc29tZSBi YXNpYwp0ZXN0cyBhbmQgcHJvdmlkZXMgYSBwbGFjZSBmb3IgZnV0dXJlIGltcHJvdmVtZW50 LgotLS0KIHNyYy90ZXN0L3JlY292ZXJ5L3QvMDQyX2xvd19sZXZlbF9iYWNrdXAucGwgfCAx MTQgKysrKysrKysrKysrKysrKysrKysKIDEgZmlsZSBjaGFuZ2VkLCAxMTQgaW5zZXJ0aW9u cygrKQogY3JlYXRlIG1vZGUgMTAwNjQ0IHNyYy90ZXN0L3JlY292ZXJ5L3QvMDQyX2xvd19s ZXZlbF9iYWNrdXAucGwKCmRpZmYgLS1naXQgYS9zcmMvdGVzdC9yZWNvdmVyeS90LzA0Ml9s b3dfbGV2ZWxfYmFja3VwLnBsIGIvc3JjL3Rlc3QvcmVjb3ZlcnkvdC8wNDJfbG93X2xldmVs X2JhY2t1cC5wbApuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwMDAwLi43NmJh YzdlMzI0Ci0tLSAvZGV2L251bGwKKysrIGIvc3JjL3Rlc3QvcmVjb3ZlcnkvdC8wNDJfbG93 X2xldmVsX2JhY2t1cC5wbApAQCAtMCwwICsxLDExNCBAQAorCisjIENvcHlyaWdodCAoYykg MjAyMywgUG9zdGdyZVNRTCBHbG9iYWwgRGV2ZWxvcG1lbnQgR3JvdXAKKworIyBUZXN0IGxv dy1sZXZlbCBiYWNrdXAgbWV0aG9kIGJ5IHVzaW5nIHBnX2JhY2t1cF9zdGFydCgpIGFuZCBw Z19iYWNrdXBfc3RvcCgpCisjIHRvIGNyZWF0ZSBiYWNrdXBzLgorCit1c2Ugc3RyaWN0Owor dXNlIHdhcm5pbmdzOworCit1c2UgRmlsZTo6Q29weSBxdyhjb3B5KTsKK3VzZSBGaWxlOjpQ YXRoIHF3KHJlbW92ZV90cmVlKTsKK3VzZSBQb3N0Z3JlU1FMOjpUZXN0OjpDbHVzdGVyOwor dXNlIFBvc3RncmVTUUw6OlRlc3Q6OlV0aWxzOwordXNlIFRlc3Q6Ok1vcmU7CisKKyMgU3Rh cnQgcHJpbWFyeSBub2RlIHdpdGggYXJjaGl2aW5nCitteSAkbm9kZV9wcmltYXJ5ID0gUG9z dGdyZVNRTDo6VGVzdDo6Q2x1c3Rlci0+bmV3KCdwcmltYXJ5Jyk7Ciskbm9kZV9wcmltYXJ5 LT5pbml0KGhhc19hcmNoaXZpbmcgPT4gMSwgYWxsb3dzX3N0cmVhbWluZyA9PiAxKTsKKyRu b2RlX3ByaW1hcnktPnN0YXJ0OworCisjIFN0YXJ0IGJhY2t1cAorbXkgJGJhY2t1cF9uYW1l ID0gJ2JhY2t1cDEnOworbXkgJHBzcWwgPSAkbm9kZV9wcmltYXJ5LT5iYWNrZ3JvdW5kX3Bz cWwoJ3Bvc3RncmVzJyk7CisKKyRwc3FsLT5xdWVyeV9zYWZlKCJTRVQgY2xpZW50X21pbl9t ZXNzYWdlcyBUTyBXQVJOSU5HIik7CiskcHNxbC0+c2V0X3F1ZXJ5X3RpbWVyX3Jlc3RhcnQo KTsKKyRwc3FsLT5xdWVyeV9zYWZlKCJzZWxlY3QgcGdfYmFja3VwX3N0YXJ0KCd0ZXN0IGxh YmVsJykiKTsKKworIyBDb3B5IGZpbGVzCitteSAkYmFja3VwX2RpciA9ICRub2RlX3ByaW1h cnktPmJhY2t1cF9kaXIoKSAuICcvJyAuICRiYWNrdXBfbmFtZTsKKworUG9zdGdyZVNRTDo6 VGVzdDo6UmVjdXJzaXZlQ29weTo6Y29weXBhdGgoCisJJG5vZGVfcHJpbWFyeS0+ZGF0YV9k aXIoKSwgJGJhY2t1cF9kaXIpOworCisjIENsZWFudXAgc29tZSBmaWxlcy9wYXRocyB0aGF0 IHNob3VsZCBub3QgYmUgaW4gdGhlIGJhY2t1cC4gVGhlcmUgaXMgbm8KKyMgYXR0ZW1wdCB0 byBhbGwgdGhlIGV4Y2x1c2lvbnMgZG9uZSBieSBwZ19iYXNlYmFja3VwIGhlcmUsIGluIHBh cnQgYmVjYXVzZQorIyB0aGV5IGFyZSBub3QgcmVxdWlyZWQsIGJ1dCBhbHNvIHRvIGtlZXAg dGhlIHRlc3Qgc2ltcGxlLgorIworIyBBbHNvIHJlbW92ZSBwZ19jb250cm9sIGJlY2F1c2Ug aXQgbmVlZHMgdG8gYmUgY29waWVkIGxhdGVyIGFuZCBpdCB3aWxsIGJlCisjIG9idmlvdXMg aWYgdGhlIGNvcHkgZmFpbHMuCit1bmxpbmsoIiRiYWNrdXBfZGlyL3Bvc3RtYXN0ZXIucGlk IikKKwlvciBCQUlMX09VVCgidW5hYmxlIHRvIHVubGluayAkYmFja3VwX2Rpci9wb3N0bWFz dGVyLnBpZCIpOwordW5saW5rKCIkYmFja3VwX2Rpci9wb3N0bWFzdGVyLm9wdHMiKQorCW9y IEJBSUxfT1VUKCJ1bmFibGUgdG8gdW5saW5rICRiYWNrdXBfZGlyL3Bvc3RtYXN0ZXIub3B0 cyIpOwordW5saW5rKCIkYmFja3VwX2Rpci9nbG9iYWwvcGdfY29udHJvbCIpCisJb3IgQkFJ TF9PVVQoInVuYWJsZSB0byB1bmxpbmsgJGJhY2t1cF9kaXIvZ2xvYmFsL3BnX2NvbnRyb2wi KTsKKworcmVtb3ZlX3RyZWUoIiRiYWNrdXBfZGlyL3BnX3dhbCIsIHtrZWVwX3Jvb3QgPT4g MX0pCisJb3IgQkFJTF9PVVQoInVuYWJsZSB0byB1bmxpbmsgY29udGVudHMgb2YgJGJhY2t1 cF9kaXIvcGdfd2FsIik7CisKKyMgQ3JlYXRlIGEgdGFibGUgdGhhdCB3aWxsIGJlIHVzZWQg dG8gdmVyaWZ5IHRoYXQgcmVjb3Zlcnkgc3RhcnRlZCBhdCB0aGUKKyMgY29ycmVjdCBsb2Nh dGlvbiwgcmF0aGVyIHRoYW4gdGhlIGxvY2F0aW9uIHJlY29yZGVkIGluIHBnX2NvbnRyb2wK KyRwc3FsLT5xdWVyeV9zYWZlKCJjcmVhdGUgdGFibGUgY2FuYXJ5IChpZCBpbnQpIik7CisK KyMgQWR2YW5jZSB0aGUgY2hlY2twb2ludCBsb2NhdGlvbiBpbiBwZ19jb250cm9sIHBhc3Qg dGhlIGxvY2F0aW9uIHdoZXJlIHRoZQorIyBiYWNrdXAgc3RhcnRlZC4gU3dpdGNoIHRoZSBX QUwgdG8gbWFrZSBpdCByZWFsbHkgb2J2aW91cyB0aGF0IHRoZSBsb2NhdGlvbgorIyBpcyBk aWZmZXJlbnQgYW5kIHRvIHB1dCB0aGUgY2hlY2twb2ludCBpbiBhIG5ldyBXQUwgc2VnbWVu dC4KKyRwc3FsLT5xdWVyeV9zYWZlKCJzZWxlY3QgcGdfc3dpdGNoX3dhbCgpIik7CiskcHNx bC0+cXVlcnlfc2FmZSgiY2hlY2twb2ludCIpOworCisjIENvcHkgcGdfY29udHJvbCBsYXN0 IHNvIGl0IGNvbnRhaW5zIHRoZSBuZXcgY2hlY2twb2ludAorY29weSgkbm9kZV9wcmltYXJ5 LT5kYXRhX2RpcigpIC4gJy9nbG9iYWwvcGdfY29udHJvbCcsCisJCSIkYmFja3VwX2Rpci9n bG9iYWwvcGdfY29udHJvbCIpCisJb3IgQkFJTF9PVVQoInVuYWJsZSB0byBjb3B5IGdsb2Jh bC9wZ19jb250cm9sIik7CisKKyMgU3RvcCBiYWNrdXAgYW5kIGdldCBiYWNrdXBfbGFiZWwK K215ICRiYWNrdXBfbGFiZWwgPSAkcHNxbC0+cXVlcnlfc2FmZSgic2VsZWN0IGxhYmVsZmls ZSBmcm9tIHBnX2JhY2t1cF9zdG9wKCkiKTsKKworIyBSYXRoZXIgdGhhbiB3cml0aW5nIG91 dCBiYWNrdXBfbGFiZWwsIHRyeSB0byByZWNvdmVyIHRoZSBiYWNrdXAgd2l0aG91dAorIyBi YWNrdXBfbGFiZWwgdG8gZGVtb25zdHJhdGUgdGhhdCByZWNvdmVyeSB3aWxsIG5vdCB3b3Jr IGNvcnJlY3RseSB3aXRob3V0IGl0LAorIyBpLmUuIHRoZSBjYW5hcnkgdGFibGUgd2lsbCBi ZSBtaXNzaW5nIGFuZCB0aGUgY2x1c3RlciB3aWxsIGJlIGNvcnJ1cHQuIFByb3ZpZGUKKyMg b25seSB0aGUgV0FMIHNlZ21lbnQgdGhhdCByZWNvdmVyeSB3aWxsIHRoaW5rIGl0IG5lZWRz LgorIworIyBUaGUgcG9pbnQgb2YgdGhpcyB0ZXN0IGlzIHRvIGV4cGxpY2l0bHkgZGVtb25z dHJhdGUgdGhhdCBiYWNrdXBfbGFiZWwgaXMKKyMgYmVpbmcgdXNlZCBpbiBhIGxhdGVyIHRl c3QgdG8gZ2V0IHRoZSBjb3JyZWN0IHJlY292ZXJ5IGluZm8uCitteSAkbm9kZV9yZXBsaWNh ID0gUG9zdGdyZVNRTDo6VGVzdDo6Q2x1c3Rlci0+bmV3KCdyZXBsaWNhX2ZhaWwnKTsKKyRu b2RlX3JlcGxpY2EtPmluaXRfZnJvbV9iYWNrdXAoJG5vZGVfcHJpbWFyeSwgJGJhY2t1cF9u YW1lKTsKK215ICRjYW5hcnlfcXVlcnkgPSAic2VsZWN0IGNvdW50KCopIGZyb20gcGdfY2xh c3Mgd2hlcmUgcmVsbmFtZSA9ICdjYW5hcnknIjsKKworY29weSgkbm9kZV9wcmltYXJ5LT5h cmNoaXZlX2RpcigpIC4gJy8wMDAwMDAwMTAwMDAwMDAwMDAwMDAwMDMnLAorCQkkbm9kZV9y ZXBsaWNhLT5kYXRhX2RpcigpIC4gJy9wZ193YWwvMDAwMDAwMDEwMDAwMDAwMDAwMDAwMDAz JykKKwlvciBCQUlMX09VVCgidW5hYmxlIHRvIGNvcHkgMDAwMDAwMDEwMDAwMDAwMDAwMDAw MDAzIik7CisKKyRub2RlX3JlcGxpY2EtPnN0YXJ0OworCitvaygkbm9kZV9yZXBsaWNhLT5z YWZlX3BzcWwoJ3Bvc3RncmVzJywgJGNhbmFyeV9xdWVyeSkgPT0gMCwKKwknY2FuYXJ5IGlz IG1pc3NpbmcnKTsKKworIyBDaGVjayBsb2cgdG8gZW5zdXJlIGNyYXNoIHJlY292ZXJ5IHdh cyB1c2VkIGJlY2F1c2UgYmFja3VwIGxhYmVsIHdhcyBtaXNzaW5nCitvaygkbm9kZV9yZXBs aWNhLT5sb2dfY29udGFpbnMoCisJJ2RhdGFiYXNlIHN5c3RlbSB3YXMgbm90IHByb3Blcmx5 IHNodXQgZG93bjsgYXV0b21hdGljIHJlY292ZXJ5IGluIHByb2dyZXNzJyksCisJJ3Zlcmlm eSBiYWNrdXAgcmVjb3ZlcnkgcGVyZm9ybWVkIHdpdGggY3Jhc2ggcmVjb3ZlcnknKTsKKwor JG5vZGVfcmVwbGljYS0+dGVhcmRvd25fbm9kZSgpOworJG5vZGVfcmVwbGljYS0+Y2xlYW5f bm9kZSgpOworCisjIFNhdmUgYmFja3VwX2xhYmVsIGludG8gdGhlIGJhY2t1cCBkaXJlY3Rv cnkgYW5kIHJlY292ZXIgdXNpbmcgdGhlIHByaW1hcnkncworIyBhcmNoaXZlLiBUaGlzIHRp bWUgcmVjb3Zlcnkgd2lsbCBzdWNjZWVkIGFuZCB0aGUgY2FuYXJ5IHRhYmxlIHdpbGwgYmUK KyMgcHJlc2VudC4KK2FwcGVuZF90b19maWxlKCIkYmFja3VwX2Rpci9iYWNrdXBfbGFiZWwi LCAkYmFja3VwX2xhYmVsKTsKKworJG5vZGVfcmVwbGljYSA9IFBvc3RncmVTUUw6OlRlc3Q6 OkNsdXN0ZXItPm5ldygncmVwbGljYV9zdWNjZXNzJyk7Ciskbm9kZV9yZXBsaWNhLT5pbml0 X2Zyb21fYmFja3VwKCRub2RlX3ByaW1hcnksICRiYWNrdXBfbmFtZSwKKwloYXNfcmVzdG9y aW5nID0+IDEpOworJG5vZGVfcmVwbGljYS0+c3RhcnQ7CisKK29rKCRub2RlX3JlcGxpY2Et PnNhZmVfcHNxbCgncG9zdGdyZXMnLCAkY2FuYXJ5X3F1ZXJ5KSA9PSAxLAorCSdjYW5hcnkg aXMgcHJlc2VudCcpOworCisjIENoZWNrIGxvZyB0byBlbnN1cmUgYmFja3VwX2xhYmVsIHdh cyB1c2VkIGZvciByZWNvdmVyeQorb2soJG5vZGVfcmVwbGljYS0+bG9nX2NvbnRhaW5zKCdz dGFydGluZyBiYWNrdXAgcmVjb3Zlcnkgd2l0aCByZWRvIExTTicpLAorCSd2ZXJpZnkgYmFj a3VwIHJlY292ZXJ5IHBlcmZvcm1lZCB3aXRoIGJhY2t1cF9sYWJlbCcpOworCitkb25lX3Rl c3RpbmcoKTsKLS0gCjIuMzQuMQoK --------------hMJSX9YBRgQL63k90JTy0Qrw--