MIME-Version: 1.0
From: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Date: Fri, 20 Feb 2026 18:55:33 +0100
Message-ID: <CAO6_Xqo3co3BuUVEVzkaBVw9LidBgeeQ_2hfxeLMQcXwovB3GQ@mail.gmail.com>
Subject: Shutdown indefinitely stuck due to unflushed FPI_FOR_HINT record
To: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Content-Type: multipart/mixed; boundary="00000000000060ade5064b4521f1"
Archived-At: <https://www.postgresql.org/message-id/CAO6_Xqo3co3BuUVEVzkaBVw9LidBgeeQ_2hfxeLMQcXwovB3GQ%40mail.gmail.com>
Precedence: bulk

--00000000000060ade5064b4521f1
Content-Type: text/plain; charset="UTF-8"

Hi,

Shutdown may be indefinitely stuck under the following circumstances:
- Data checksum is enabled (needed to generate FPI_FOR_HINT record)
- A logical replication walsender is running
- A select in an explicit ongoing transaction pruned a heap page and
logged a FPI_FOR_HINT record. This record is likely going to be a
contrecord and start a new page.

Starting the shutdown will kill this ongoing transaction. Since the
transaction doesn't have an allocated xid, the FPI_FOR_HINT record
will be left unflushed.

When the checkpointer calls ShutdownXLOG(), all walsenders will be
notified to stop. However, the logical replication walsender will be
stuck in an infinite loop, trying to read this unflushed record and
never reaching the stop state, blocking the whole shutdown sequence.

This can be reproduced with the following script (this assumes
`pgbench -i` was run to create pgbench_accounts and a running logical
replication walsender):

TRUNCATE pgbench_accounts;
-- Completely fill the first heap page
INSERT INTO pgbench_accounts SELECT *, *, *, '' FROM generate_series(0, 62);
-- This should tag the page's metadata as full
BEGIN;
UPDATE pgbench_accounts SET bid=4 where aid=1;
ROLLBACK;
-- Force checkpoint so next change will be a FPW
CHECKPOINT;
-- Open an explicit transaction
BEGIN;
-- Select will do an opportunistic pruning, find nothing to prune but
will still unset the page full flag, writing a FPI_FOR_HINT
SELECT ctid, * FROM pgbench_accounts WHERE aid=2;

Then shutdown the database with 'pg_ctl stop' with the transaction
left opened. The shutdown will be stuck and the logical replication
walsender will be stuck at 100% CPU.

I've managed to reproduce this issue on 14 and the current HEAD.

The attached (tentative) patch fixes the issue by flushing all records
before signaling walsenders to stop. At that point, all backends
should have been killed, so flushing leftover records felt like a
correct approach.

Regards,
Anthonin Bonnefoy

--00000000000060ade5064b4521f1
Content-Type: application/octet-stream; 
	name="v1-0001-Fix-stuck-shutdown-due-to-unflushed-records.patch"
Content-Disposition: attachment; 
	filename="v1-0001-Fix-stuck-shutdown-due-to-unflushed-records.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mlv6tg3r0>
X-Attachment-Id: f_mlv6tg3r0

RnJvbSBmNjJiNmI0NTU5NGI0ZDU4ZmZlNzM5Y2E0MmVjZDNhY2E2NjA1YzRjIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBBbnRob25pbiBCb25uZWZveSA8YW50aG9uaW4uYm9ubmVmb3lA
ZGF0YWRvZ2hxLmNvbT4KRGF0ZTogRnJpLCAyMCBGZWIgMjAyNiAxODoxNToxMiArMDEwMApTdWJq
ZWN0OiBGaXggc3R1Y2sgc2h1dGRvd24gZHVlIHRvIHVuZmx1c2hlZCByZWNvcmRzCgpTaHV0ZG93
biBzZXF1ZW5jZSBtYXkgYmUgc3R1Y2sgaW5kZWZpbml0ZWx5IHVuZGVyIHRoZSBmb2xsb3dpbmcK
Y2lyY3Vtc3RhbmNlczoKLSBEYXRhIGNoZWNrc3VtcyBpcyBlbmFibGVkCi0gQSBsb2dpY2FsIHJl
cGxpY2F0aW9uIHdhbHNlbmRlciBpcyBydW5uaW5nCi0gQSBzZWxlY3QgaW4gYW4gZXhwbGljaXQg
b25nb2luZyB0cmFuc2FjdGlvbiBwcnVuZWQgYSBoZWFwIHBhZ2UgYW5kCiAgbG9nZ2VkIGEgRlBJ
X0ZPUl9ISU5UIHJlY29yZC4gVGhpcyByZWNvcmQgaXMgbGlrZWx5IGdvaW5nIHRvIGJlIGEKICBj
b250cmVjb3JkIGFuZCBzdGFydCBhIG5ldyBwYWdlLgoKU3RhcnRpbmcgdGhlIHNodXRkb3duIHdp
bGwga2lsbCB0aGlzIG9uZ29pbmcgdHJhbnNhY3Rpb24uIFNpbmNlIHRoZQp0cmFuc2FjdGlvbiBk
b2Vzbid0IGhhdmUgYW4gYWxsb2NhdGVkIHhpZCwgdGhlIEZQSV9GT1JfSElOVCByZWNvcmQgd2ls
bApiZSBsZWZ0IHVuZmx1c2hlZC4KCldoZW4gdGhlIGNoZWNrcG9pbnRlciBzdGFydHMgU2h1dGRv
d25YTE9HKCksIGFsbCB3YWxzZW5kZXJzIHdpbGwgYmUKbm90aWZpZWQgdG8gc3RvcC4gSG93ZXZl
ciwgdGhlIGxvZ2ljYWwgcmVwbGljYXRpb24gd2Fsc2VuZGVyIHdpbGwgYmUKc3R1Y2sgaW4gYW4g
aW5maW5pdGUgbG9vcCwgdHJ5aW5nIHRvIHJlYWQgdGhpcyB1bmZsdXNoZWQgcmVjb3JkLCBuZXZl
cgpyZWFjaGluZyB0aGUgc3RvcCBzdGF0ZSBhbmQgYmxvY2tpbmcgdGhlIHdob2xlIHNodXRkb3du
IHNlcXVlbmNlLgoKVGhpcyBwYXRjaCBmaXhlcyB0aGUgaXNzdWUgYnkgZmx1c2hpbmcgYWxsIHJl
Y29yZHMgYmVmb3JlIHNpZ25hbGluZwp3YWxzZW5kZXJzIHRvIHN0b3AuCi0tLQogc3JjL2JhY2tl
bmQvYWNjZXNzL3RyYW5zYW0veGxvZy5jIHwgMTEgKysrKysrKysrKysKIDEgZmlsZSBjaGFuZ2Vk
LCAxMSBpbnNlcnRpb25zKCspCgpkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvYWNjZXNzL3RyYW5z
YW0veGxvZy5jIGIvc3JjL2JhY2tlbmQvYWNjZXNzL3RyYW5zYW0veGxvZy5jCmluZGV4IDEzZWM2
MjI1Yjg1Li5hYTQ5MDE3NmFhZiAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQvYWNjZXNzL3RyYW5z
YW0veGxvZy5jCisrKyBiL3NyYy9iYWNrZW5kL2FjY2Vzcy90cmFuc2FtL3hsb2cuYwpAQCAtNjcy
Nyw2ICs2NzI3LDggQEAgR2V0TGFzdFNlZ1N3aXRjaERhdGEoWExvZ1JlY1B0ciAqbGFzdFN3aXRj
aExTTikKIHZvaWQKIFNodXRkb3duWExPRyhpbnQgY29kZSwgRGF0dW0gYXJnKQogeworCVhMb2dS
ZWNQdHIJV3JpdGVScXN0UHRyOworCiAJLyoKIAkgKiBXZSBzaG91bGQgaGF2ZSBhbiBhdXggcHJv
Y2VzcyByZXNvdXJjZSBvd25lciB0byB1c2UsIGFuZCB3ZSBzaG91bGQgbm90CiAJICogYmUgaW4g
YSB0cmFuc2FjdGlvbiB0aGF0J3MgaW5zdGFsbGVkIHNvbWUgb3RoZXIgcmVzb3duZXIuCkBAIC02
NzQwLDYgKzY3NDIsMTUgQEAgU2h1dGRvd25YTE9HKGludCBjb2RlLCBEYXR1bSBhcmcpCiAJZXJl
cG9ydChJc1Bvc3RtYXN0ZXJFbnZpcm9ubWVudCA/IExPRyA6IE5PVElDRSwKIAkJCShlcnJtc2co
InNodXR0aW5nIGRvd24iKSkpOwogCisJLyoKKwkgKiBXZSBtYXkgaGF2ZSB1bmZsdXNoZWQgcmVj
b3JkcywgbWFrZSBzdXJlIGV2ZXJ5dGhpbmcgaXMgZmx1c2hlZCBiZWZvcmUKKwkgKiBzdG9wcGlu
ZyB0aGUgd2Fsc2VuZGVycy4KKwkgKi8KKwlTcGluTG9ja0FjcXVpcmUoJlhMb2dDdGwtPmluZm9f
bGNrKTsKKwlXcml0ZVJxc3RQdHIgPSBYTG9nQ3RsLT5Mb2d3cnRScXN0LldyaXRlOworCVNwaW5M
b2NrUmVsZWFzZSgmWExvZ0N0bC0+aW5mb19sY2spOworCVhMb2dGbHVzaChXcml0ZVJxc3RQdHIp
OworCiAJLyoKIAkgKiBTaWduYWwgd2Fsc2VuZGVycyB0byBtb3ZlIHRvIHN0b3BwaW5nIHN0YXRl
LgogCSAqLwotLSAKMi41Mi4wCgo=
--00000000000060ade5064b4521f1--