Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xTKzb-00000000GnQ-15gz for pgsql-hackers@arkaria.postgresql.org; Tue, 17 Nov 2026 15:21:07 +0000 Received: from localhost ([127.0.0.1]:37572 helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xTKzZ-00000005mOz-15W0 for pgsql-hackers@arkaria.postgresql.org; Tue, 17 Nov 2026 15:21:05 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]:52450) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xTKzY-00000005mOq-2sYL for pgsql-hackers@lists.postgresql.org; Tue, 17 Nov 2026 15:21:04 +0000 Received: from mail-wm2-x09.google.com ([2a00:1450:4864:31::9]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1xTKzW-00000000AJR-0nW9 for pgsql-hackers@lists.postgresql.org; Tue, 17 Nov 2026 15:21:03 +0000 Received: by mail-wm2-x09.google.com with SMTP id 5b1f17b1804b1-4ad5d6045d4so19415445e9.0 for ; Tue, 17 Nov 2026 07:21:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1794928860; x=1795533660; darn=lists.postgresql.org; h=in-reply-to:from:references:cc:to:subject:user-agent:mime-version :date:message-id:content-type:from:to:cc:subject:date:message-id :reply-to:content-type; bh=6EK0257IZiMpjsQ+MIrWWPry8a3l+ivoAuKgaXyeGxU=; b=SQl8RyBvJuWBoIVD8qqivFq+V3jUBHBpCymC/BDqfa2Wyk3ul2JyzkbrDBu4MnTO+f YRO1SAktHeGQq6LcpONOyFQi+8CHVtVGi30GU8uWZUjnvRSJ/z0wG5QQHP0nftTi0z56 i6vRf5cBQpSXLHItHdLS7umfeh+FMgBz1B6CCi6Mo3POQh6IHg4vHjlkYnawBDsJTfvl nfebRjpHQxkhear4HjVSBTSpKKHb1LDmifjMRPbt9PiVUZROGCfaXPjlEUyDC40DEp24 Gw7r3vdYSATM3CoqCEhlmm8s0hm5gRlIm8N1TEWrT9H1mXza9oZtPR2Z/OaSl4BxmGSZ 7Tlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794928860; x=1795533660; h=in-reply-to:from:references:cc:to:subject:user-agent:mime-version :date:message-id:content-type:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to:content-type; bh=6EK0257IZiMpjsQ+MIrWWPry8a3l+ivoAuKgaXyeGxU=; b=XF+bnMhSc3dqS7hpIy5c2fXSX+Beze9yJihsQAH2cRvTtPziBP/YWAy+3D3WfuQjRm 8KXRZmlaW3/6PA83Vt5TtNmY4ahLGLPbCZDUshETctIPzjZbikf9vH5Sa8rsjghvHpK7 +n/gMOG/qU+2vqMzzYdklbNbjKxFIhhM+mEPMwh7hTwg09VZNBloggOV0e3N+8F+BvZC CjMABXS5lbTF1HQ5IFz16hypoo1npjx6LsnPsXklgeb/AVBKL8wW1oxDZS/mY9MnJM/O YklbVCwhmsE+RmaxsHQY/3173vX+LWG27v2khYEaS2tWg3pKN44AqJinua0GMGEMths5 SRYw== X-Forwarded-Encrypted: i=1; AL+hErH59etYToiAgiDrmLuubO9T16nr0M/L+d/JzRBT36+9tyr4ANRdXUAXmWEtVCicG2vz2nCepB9eH275k5k0@lists.postgresql.org X-Gm-Message-State: APuU4Z/Sowa3E3Sz97CDpoNNpZsWjufMJE8JkTPL4YAullA9SQ2fAiNF EPwxk4/0IENawn1QTK0njhyt9JwQ7ErfX8u2Otd+IUGQrHqR5IHHWrr1 X-Gm-Gg: AYBFou0+63QcrODT2OJUSKPGgxqz0xYGsGeoEY62eWIh7YzeA14Yzz99rWr3h0XakCs M75meP6bFvXTqWMdnMa2PEb9D90jrgpsB6f/BtJuzUHolL8o0Ficsy3QB/bKRBw59KMGCHrYeVy 9CcpyFiQzUEtSh0mxwZ8CQHptGRdUS6HTjeCgyGdMD4MqIhNBZHBZCi3pXRkw2znPKVJG3ofHO1 qbk1i0xY8CvUSv9ahW0mvfMnJrwWFIqohnhH48ka59Y+6XoCBbpN5z56iDpgqwRhzjioAEFTUMs OptdI4z8vniNE5D4MOwo/2xf2cvbKPhUTIhMTlPr5Enq5CxHCvsW4eQmDBIF7RLpLLSd73hp4rA 1VDq1iEo9DlMgsNn0y0UnNW//Aupjv6HwlpRDEJW2Zl2PUcVWGTqwBEbIxiuLiRwJz2kHyt60Bd UYlWthY0VcY2wPG07tltPhGAKdLW2Xr4aCZ0hEbdbdLxx7w0vBXk3xWCBOACWeV2flqRyAStidd kJlID2K03M2FVCQUERFY/X87xpcdbFYDuDe44BiqY75+xRCyD2eEsSnvUYreqbTKf9UgeI7+ZA3 mAeAVLebX6EYndLph3ZeeV2dseQfxtHPAij378BzzQEvGdjOXVO3bROAlS4GHV27DhOtyeoVOwS oTm4hTiy7cA8K4D/ZEjrQZa3wj38cot4A5pszKicw3pD16bd9lYv0P5zQtoA= X-Received: by 2002:a05:600c:1d06:b0:4a2:190f:cf6a with SMTP id 5b1f17b1804b1-4bb55ab3403mr284436655e9.35.1794928860216; Tue, 17 Nov 2026 07:21:00 -0800 (PST) Received: from [17.235.206.166] ([17.235.206.166]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb734b9f44sm102810135e9.1.2026.11.17.07.20.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Nov 2026 07:20:59 -0800 (PST) Content-Type: multipart/mixed; boundary="------------aCGyEus7cCkaWTjHJ2SowzKz" Message-ID: <2889116c-7da1-4ab0-b10a-3fadb99c24e7@gmail.com> Date: Tue, 17 Nov 2026 23:20:54 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: WAL_LOG CREATE DATABASE strategy broken for non-standard page layouts To: Melanie Plageman Cc: Robert Haas , Michael Paquier , Rogers Wang , pgsql-hackers@lists.postgresql.org, Matthias van de Meent , =?UTF-8?Q?=C3=81lvaro_Herrera?= , Andres Freund References: <202602021525.qv4m7oge4eca@alvherre.pgsql> <6e62d593-a8b2-415d-834d-e54ed519bb04@gmail.com> From: Zhijie Hou In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------aCGyEus7cCkaWTjHJ2SowzKz Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi, On Tue, Nov17, 2026 at 01:35 AM Melanie Plageman wrote: > On Sat, Nov 14, 2026 at 8:08 AM Zhijie Hou wrote: > > > > I just have one minor comment on the TAP test in 0001. IIUC, the > > pre-existing test in 034_create_database.pl was added by commit > > 8a8661828ae to verify that when CREATE DATABASE loads the template's > > pg_class into shared_buffers, it marks those buffers as permanent so > > that a subsequent checkpoint flushes them to disk. > > > > But with 0001, the modified test vacuuming tab_db_before_create_1 in > > template1 beforehand loads template1's pg_class into shared_buffers > > with BM_PERMANENT already set before CREATE DATABASE runs. If we > > want to keep the original regression coverage, maybe we could run > > the new test steps after the pre-existing test. > > Ah, thanks so much for catching this. I read the test and not the > commit message adding it and thought it was just trying to make sure > that tables added after the copy didn't end up in the copied database > -- which, if I think about it, would have been a silly test. I've > changed it in attached v3 to retain the coverage in the initial test. > > I also realized while editing this that I'll have to make a wrapper > version of CreateAndCopyRelationData() for the backports since it is > public. A bit annoying but I think better than the alternatives. Thanks for updating the patch. I think we also need to increase shared_buffers (e.g., to 16MB), because $node->init(allows_streaming => 1) sets shared_buffers to 1MB internally, causing the pg_class pages to be evicted from shared_buffers before CREATE TABLE runs, so the test could not reach the old bug path. BTW, while looking into why the wal_consistency_checking test didn't catch this bug, I noticed that verifyBackupPageConsistency() also masks out all the bits in VM pages via heap_mask() (somewhat similar to this bug, since it treats pd_lower..pd_upper as unused space), so the consistency check for VM pages always "passes". Once we avoid masking the VM bitmap there (see the attached draft patch for reference), the existing test catches this CREATE DB WAL_LOG bug right away: PG_TEST_EXTRA=wal_consistency_checking meson test -C build recovery/027_stream_regress So if it's worth increasing the possibility of catching similar VM bugs, maybe we could improve that as well (in a separate patch or thread). -- Best Regards, Zhijie Hou --------------aCGyEus7cCkaWTjHJ2SowzKz Content-Type: text/plain; charset=UTF-8; name="nocfbot-0001-Fix-WAL-consistency-checking-for-visibility-map-p.patch" Content-Disposition: attachment; filename*0="nocfbot-0001-Fix-WAL-consistency-checking-for-visibility-map"; filename*1="-p.patch" Content-Transfer-Encoding: base64 RnJvbSA0ZWRlNzJlOGVlNmQ3OWNkYTgyMDlhNzI1YjFiNmIwNGUwNzQzOGE0IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBaaGlqaWUgSG91IDxob3V6aGlqaWUyMkBnbWFpbC5j b20+CkRhdGU6IFR1ZSwgMTcgTm92IDIwMjYgMjI6MDY6MzggKzA4MDAKU3ViamVjdDogW1BB VENIIHYxIDEvMl0gRml4IFdBTCBjb25zaXN0ZW5jeSBjaGVja2luZyBmb3IgdmlzaWJpbGl0 eSBtYXAgcGFnZXMuCgpCb3RoIFJNX0hFQVBfSUQgYW5kIFJNX0hFQVAyX0lEIFdBTCByZWNv cmRzIHJlZ2lzdGVyIFZJU0lCSUxJVFlNQVBfRk9SS05VTQpidWZmZXJzIChzaW5jZSBhNTA3 Yjg2OTAwZiBmb3IgVk0tc2V0dGluZyByZWNvcmRzLCBhbmQgc2luY2UgZWQ2MmQyNmNhY2EK Zm9yIFZNLWNsZWFyaW5nIHJlY29yZHMpLiAgV2hlbiB3YWxfY29uc2lzdGVuY3lfY2hlY2tp bmcgaXMgZW5hYmxlZCBmb3IKaGVhcCBvciBoZWFwMiwgdmVyaWZ5QmFja3VwUGFnZUNvbnNp c3RlbmN5KCkgcHJldmlvdXNseSBwYXNzZWQgZXZlcnkKcmVnaXN0ZXJlZCBibG9jayB0byBy bWdyLnJtX21hc2sgKGhlYXBfbWFzaygpKSB3aXRob3V0IGNoZWNraW5nIGZvcmtudW0uCgpC ZWNhdXNlIGEgdmlzaWJpbGl0eSBtYXAgcGFnZSBpcyBpbml0aWFsaXplZCB3aXRoIFBhZ2VJ bml0KHBhZ2UsIEJMQ0tTWiwgMCkKYW5kIHN0b3JlcyBpdHMgYml0bWFwIGJldHdlZW4gcGRf bG93ZXIgKFNpemVPZlBhZ2VIZWFkZXJEYXRhKSBhbmQgcGRfdXBwZXIKKEJMQ0tTWiksIG1h c2tfdW51c2VkX3NwYWNlKCkgaW4gaGVhcF9tYXNrKCkgemVyb2VkIG91dCB0aGUgZW50aXJl IGJpdG1hcAppbiBib3RoIHRoZSBXQUwgYmFja3VwIGltYWdlIGFuZCB0aGUgcmVwbGF5IGlt YWdlIGJlZm9yZSBjb21wYXJpbmcgdGhlbS4KQ29uc2VxdWVudGx5LCB3YWxfY29uc2lzdGVu Y3lfY2hlY2tpbmcgc2lsZW50bHkgbWlzc2VkIHZpc2liaWxpdHkgbWFwIHBhZ2UKZGl2ZXJn ZW5jZXMgZHVyaW5nIFdBTCByZXBsYXkuCgpGaXggdmVyaWZ5QmFja3VwUGFnZUNvbnNpc3Rl bmN5KCkgdG8gbWFzayBvbmx5IHRoZSBwYWdlIExTTiBhbmQgY2hlY2tzdW0Kd2hlbiBmb3Jr bnVtIGlzIFZJU0lCSUxJVFlNQVBfRk9SS05VTSwgYXZvaWRpbmcgYW4gQUJJIGJyZWFrIHRv ClJtZ3JEYXRhLnJtX21hc2suCi0tLQogc3JjL2JhY2tlbmQvYWNjZXNzL3RyYW5zYW0veGxv Z3JlY292ZXJ5LmMgfCAxOCArKysrKysrKysrKysrKystLS0KIDEgZmlsZSBjaGFuZ2VkLCAx NSBpbnNlcnRpb25zKCspLCAzIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3NyYy9iYWNr ZW5kL2FjY2Vzcy90cmFuc2FtL3hsb2dyZWNvdmVyeS5jIGIvc3JjL2JhY2tlbmQvYWNjZXNz L3RyYW5zYW0veGxvZ3JlY292ZXJ5LmMKaW5kZXggNTJhYzZkNTg0MjMuLjQ5MGU1MWYyNmU1 IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9hY2Nlc3MvdHJhbnNhbS94bG9ncmVjb3Zlcnku YworKysgYi9zcmMvYmFja2VuZC9hY2Nlc3MvdHJhbnNhbS94bG9ncmVjb3ZlcnkuYwpAQCAt MzAsNiArMzAsNyBAQAogI2luY2x1ZGUgPHN5cy90aW1lLmg+CiAjaW5jbHVkZSA8dW5pc3Rk Lmg+CiAKKyNpbmNsdWRlICJhY2Nlc3MvYnVmbWFzay5oIgogI2luY2x1ZGUgImFjY2Vzcy90 aW1lbGluZS5oIgogI2luY2x1ZGUgImFjY2Vzcy90cmFuc2FtLmgiCiAjaW5jbHVkZSAiYWNj ZXNzL3hhY3QuaCIKQEAgLTI1NDcsMTAgKzI1NDgsMjEgQEAgdmVyaWZ5QmFja3VwUGFnZUNv bnNpc3RlbmN5KFhMb2dSZWFkZXJTdGF0ZSAqcmVjb3JkKQogCQkJCQkgZXJybXNnX2ludGVy bmFsKCIlcyIsIHJlY29yZC0+ZXJyb3Jtc2dfYnVmKSkpOwogCiAJCS8qCi0JCSAqIElmIG1h c2tpbmcgZnVuY3Rpb24gaXMgZGVmaW5lZCwgbWFzayBib3RoIHRoZSBwcmltYXJ5IGFuZCBy ZXBsYXkKLQkJICogaW1hZ2VzCisJCSAqIE1hc2sgYm90aCB0aGUgcHJpbWFyeSBhbmQgcmVw bGF5IGltYWdlcyBiZWZvcmUgY29tcGFyaW5nIHRoZW0uCisJCSAqCisJCSAqIFZpc2liaWxp dHkgbWFwIHBhZ2VzIGFyZSBtb2RpZmllZCBieSBSTV9IRUFQX0lEIGFuZCBSTV9IRUFQMl9J RAorCQkgKiByZWNvcmRzLCBidXQgZG8gbm90IGZvbGxvdyB0aGUgc3RhbmRhcmQgaGVhcCBw YWdlIGxheW91dDogdGhlaXIKKwkJICogYml0bWFwIG9jY3VwaWVzIHRoZSBieXRlcyBiZXR3 ZWVuIHBkX2xvd2VyIGFuZCBwZF91cHBlciAoc2VlCisJCSAqIHZpc2liaWxpdHltYXAuYyks IHdoaWNoIGhlYXBfbWFzaygpIHdvdWxkIHdpcGUgdmlhCisJCSAqIG1hc2tfdW51c2VkX3Nw YWNlKCkuICBPbmx5IHRoZSBMU04gYW5kIGNoZWNrc3VtIGNhbiBsZWdpdGltYXRlbHkKKwkJ ICogZGlmZmVyIG9uIGEgdmlzaWJpbGl0eSBtYXAgcGFnZS4KIAkJICovCi0JCWlmIChybWdy LnJtX21hc2sgIT0gTlVMTCkKKwkJaWYgKGZvcmtudW0gPT0gVklTSUJJTElUWU1BUF9GT1JL TlVNKQorCQl7CisJCQltYXNrX3BhZ2VfbHNuX2FuZF9jaGVja3N1bShyZXBsYXlfaW1hZ2Vf bWFza2VkKTsKKwkJCW1hc2tfcGFnZV9sc25fYW5kX2NoZWNrc3VtKHByaW1hcnlfaW1hZ2Vf bWFza2VkKTsKKwkJfQorCQllbHNlIGlmIChybWdyLnJtX21hc2sgIT0gTlVMTCkKIAkJewog CQkJcm1nci5ybV9tYXNrKHJlcGxheV9pbWFnZV9tYXNrZWQsIGJsa25vKTsKIAkJCXJtZ3Iu cm1fbWFzayhwcmltYXJ5X2ltYWdlX21hc2tlZCwgYmxrbm8pOwotLSAKMi41NC4wIChBcHBs ZSBHaXQtMTU3KQoK --------------aCGyEus7cCkaWTjHJ2SowzKz--