MIME-Version: 1.0
References: 
 <CAHGQGwHFKF+x4E+SqedMCnmLCitxjTUUtSyL_+mMeuq-GbEt6w@mail.gmail.com>
 <CAO6_Xqp+ADb6KZVWLMALu3xmwVUEO8S1EiCnp38mG6BrHrEnuA@mail.gmail.com>
 <CAO6_XqqKDV+AuP=Gf4kRKPqzyYTsOyGd3LE8Jqkwi7EMPJpbhA@mail.gmail.com>
 <CAHGQGwHc5yH4Nxp59KXJP0kAr61j3W7QeSKT2HxVjZa3OrLzmg@mail.gmail.com>
 <CAO6_Xqq1h6kggb1o206rgouPS0H5jnjahzZ0We-9ggnBjB2JsA@mail.gmail.com>
 <CAHGQGwFJnNUOMiW9wR-2WjSKzzj0wV8p55J8bnJ6mik=z0oFPQ@mail.gmail.com>
 <2631a3c3-5e60-4a1a-9e20-377024322602@gmail.com>
 <CAO6_Xqq73TPa3M6nQ7RqRhKkcphy1JX7aNGTYy-x_Sn+6a8Z_Q@mail.gmail.com>
 <CAHGQGwGvnpN=2bo+F7H90YLFcx9=SazwLkcx+0gEcrbQy5NVZg@mail.gmail.com>
 <CAHGQGwECpyJtMqkCEvyqgZDiwAeMj3RKobui7jONrDd35W0x3Q@mail.gmail.com>
 <vzguaguldbcyfbyuq76qj7hx5qdr5kmh67gqkncyb2yhsygrdt@dfhcpteqifux>
In-Reply-To: <vzguaguldbcyfbyuq76qj7hx5qdr5kmh67gqkncyb2yhsygrdt@dfhcpteqifux>
From: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Date: Thu, 12 Mar 2026 15:07:59 +0100
Message-ID: 
 <CAO6_Xqq=-KnxrFQvmyF++XH4ngtXVhcDB979rEn6SPtUhSmNYg@mail.gmail.com>
Subject: Re: Shutdown indefinitely stuck due to unflushed FPI_FOR_HINT record
To: Andres Freund <andres@anarazel.de>
Cc: Fujii Masao <masao.fujii@gmail.com>,
 Alexander Lakhin <exclusion@gmail.com>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Content-Type: multipart/mixed; boundary="00000000000056230a064cd44820"
Archived-At: 
 <https://www.postgresql.org/message-id/CAO6_Xqq%3D-KnxrFQvmyF%2B%2BXH4ngtXVhcDB979rEn6SPtUhSmNYg%40mail.gmail.com>
Precedence: bulk

--00000000000056230a064cd44820
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Mar 10, 2026 at 6:11=E2=80=AFPM Andres Freund <andres@anarazel.de> =
wrote:
> I'm pretty sure this is not correct as-is, it suffers from the same issue=
 as
> https://postgr.es/m/vf4hbwrotvhbgcnknrqmfbqlu75oyjkmausvy66ic7x7vuhafx%40=
e4rvwavtjswo
> I.e. it is not safe to use GetXLogInsertRecPtr() to determine up to where=
 to
> flush to, due to page boundaries.

I've managed to reproduce this issue by ensuring the FPI_FOR_HINT
record finishes at the end of a page with the following script (might
need some adjustment if the record sizes are different):

DROP TABLE IF EXISTS test_insert_rec_ptr;
CREATE TABLE test_insert_rec_ptr(aid int, data text) WITH
(autovacuum_enabled =3D false);
INSERT INTO test_insert_rec_ptr SELECT *, repeat('a', 100) FROM
generate_series(0, 57);
-- This should tag the page as full
BEGIN; UPDATE test_insert_rec_ptr SET aid=3D2 where aid=3D1; ROLLBACK;
CHECKPOINT;
-- Start with a fresh file
SELECT pg_switch_wal();
-- Our FPI_FOR_HINT writes 8193 bytes
-- With the long header,  the first  page has 8152 bytes available
-- With the short header, the second page has 8168 bytes available
-- We want our FPI_FOR_HINT to finish at the end of the second page
(+/- 8 bytes of alignment)
-- We need to write the first 25 bytes (or 32 with alignment) in the first =
page
-- For that, we need to write 8120 bytes of WAL records
BEGIN;
-- 264 bytes of FPW
INSERT INTO test_insert_rec_ptr VALUES(1);
-- 74 * 104 bytes
INSERT INTO test_insert_rec_ptr SELECT *, repeat('a', 44) FROM
generate_series(1, 74);
-- 108 bytes
INSERT INTO test_insert_rec_ptr VALUES(1, repeat('a', 48));
-- 46 bytes
COMMIT;
-- 264 + 74 * 104 + 46 + 108 =3D 8114 bytes, which will round up to 8120
with alignment
-- FPI_FOR_HINT record should be at 0x1FE0
BEGIN; SELECT * FROM test_insert_rec_ptr WHERE aid=3D2; ROLLBACK;

As far as I can tell, the only impact it has is to complain about the
write request being too far:
LOG:  request to flush past end of generated WAL; request 0/01604018,
current position 0/01604000
ERROR:  xlog flush request 0/01604018 is not satisfied --- flushed
only to 0/01604000

To avoid this issue, it sounds like we need something to use
XLogBytePosToEndRecPtr instead of XLogBytePosToRecPtr to convert the
byte position? With XLogBytePosToRecPtr(), the flush request would
stop at 01604000 instead of going to the next page with 01604018.

In the attached patch, I've added a GetXLogInsertEndRecPtr() function
which is similar to GetXLogInsertRecPtr(), except it uses
XLogBytePosToEndRecPtr() to stop at the page boundary.
There was also another XLogFlush(GetXLogWriteRecPtr()) call in
syncutils.c, so I replaced both calls with
XLogFlush(GetXLogInsertEndRecPtr()).

Regards,
Anthonin Bonnefoy

--00000000000056230a064cd44820
Content-Type: application/octet-stream;
	name="v1-0001-Fix-flushing-record-ending-at-page-boundary.patch"
Content-Disposition: attachment;
	filename="v1-0001-Fix-flushing-record-ending-at-page-boundary.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mmnjd6d00>
X-Attachment-Id: f_mmnjd6d00

RnJvbSA1Y2ZkYjAyZWY2ZDA2OTdlNTY3MDM1NjM0NWZjODlhZjc4ZDFlZWI5IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBBbnRob25pbiBCb25uZWZveSA8YW50aG9uaW4uYm9ubmVmb3lA
ZGF0YWRvZ2hxLmNvbT4KRGF0ZTogVGh1LCAxMiBNYXIgMjAyNiAxNDoyOToyMSArMDEwMApTdWJq
ZWN0OiBGaXggZmx1c2hpbmcgcmVjb3JkIGVuZGluZyBhdCBwYWdlIGJvdW5kYXJ5CgpJbiA2ZWVk
YjJhNWZkLCBhIGNhbGwgdG8gWExvZ0ZsdXNoKEdldFhMb2dJbnNlcnRSZWNQdHIoKSkgaGFzIGJl
ZW4gYWRkZWQKdG8gYWxsb3cgd2Fsc2VuZGVyIHRvIGZsdXNoIHRoZSBsYXRlc3QgV0FMIHJlY29y
ZC4gSG93ZXZlciwgaWYgdGhlIGxhc3QKcmVjb3JkIGlzIGF0IHRoZSBlbmQgb2YgYSBwYWdlLCBH
ZXRYTG9nSW5zZXJ0UmVjUHRyKCkgd2lsbCByZXR1cm4gdGhlCnN0YXJ0IHBvc2l0aW9uIGZvciB0
aGUgbmV4dCByZWNvcmQsIHdoaWNoIHdpbGwgYmUgbG9jYXRlZCBpbiB0aGUgbmV4dApwYWdlLCBh
ZnRlciB0aGUgcGFnZSBoZWFkZXIuCgpYTG9nSW5zZXJ0IHdpbGwgY29tcGxhaW4gd2l0aCBhICd4
bG9nIGZsdXNoIHJlcXVlc3QgMC8wMzYwNDAxOCBpcyBub3QKc2F0aXNmaWVkIC0tLSBmbHVzaGVk
IG9ubHkgdG8gMC8wMzYwNDAwMCcgZXJyb3IsIGFzIHRoZSBmbHVzaCByZXF1ZXN0CnRyaWVzIHRv
IHdyaXRlIFdBTCB0aGF0IGhhc24ndCBiZWVuIHJlc2VydmVkIHlldC4KClRoaXMgcGF0Y2ggZml4
ZXMgdGhlIGlzc3VlIGJ5IGludHJvZHVjaW5nIGFuZCB1c2luZyBhCkdldFhMb2dJbnNlcnRFbmRS
ZWNQdHIoKSB3aGljaCBzdG9wcyBhdCB0aGUgcGFnZSBib3VuZGFyeSwgaW5zdGVhZCBvZgp0aGUg
YmVnaW5uaW5nIG9mIHRoZSBuZXh0IHBhZ2UuCi0tLQogc3JjL2JhY2tlbmQvYWNjZXNzL3RyYW5z
YW0veGxvZy5jICAgICAgICAgICB8IDE4ICsrKysrKysrKysrKysrKysrKwogc3JjL2JhY2tlbmQv
cmVwbGljYXRpb24vbG9naWNhbC9zeW5jdXRpbHMuYyB8ICAyICstCiBzcmMvYmFja2VuZC9yZXBs
aWNhdGlvbi93YWxzZW5kZXIuYyAgICAgICAgIHwgIDIgKy0KIHNyYy9pbmNsdWRlL2FjY2Vzcy94
bG9nLmggICAgICAgICAgICAgICAgICAgfCAgMSArCiA0IGZpbGVzIGNoYW5nZWQsIDIxIGluc2Vy
dGlvbnMoKyksIDIgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvYWNjZXNz
L3RyYW5zYW0veGxvZy5jIGIvc3JjL2JhY2tlbmQvYWNjZXNzL3RyYW5zYW0veGxvZy5jCmluZGV4
IGI5YjY3OGYzNzIyLi45ZmQ5MDYzNmVlMSAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQvYWNjZXNz
L3RyYW5zYW0veGxvZy5jCisrKyBiL3NyYy9iYWNrZW5kL2FjY2Vzcy90cmFuc2FtL3hsb2cuYwpA
QCAtOTU5NSw2ICs5NTk1LDI0IEBAIEdldFhMb2dJbnNlcnRSZWNQdHIodm9pZCkKIAlyZXR1cm4g
WExvZ0J5dGVQb3NUb1JlY1B0cihjdXJyZW50X2J5dGVwb3MpOwogfQogCisvKgorICogTGlrZSBH
ZXRYTG9nSW5zZXJ0UmVjUHRyLCBidXQgaWYgdGhlIHBvc2l0aW9uIGlzIGF0IGEgcGFnZSBib3Vu
ZGFyeSwgcmV0dXJucworICogYSBwb2ludGVyIHRvIHRoZSBiZWdpbm5pbmcgb2YgdGhlIHBhZ2Ug
KGllLiBiZWZvcmUgcGFnZSBoZWFkZXIpLCBub3QgdG8gd2hlcmUKKyAqIHRoZSBmaXJzdCB4bG9n
IHJlY29yZCBvbiB0aGF0IHBhZ2Ugd291bGQgZ28gdG8uCisgKi8KK1hMb2dSZWNQdHIKK0dldFhM
b2dJbnNlcnRFbmRSZWNQdHIodm9pZCkKK3sKKwlYTG9nQ3RsSW5zZXJ0ICpJbnNlcnQgPSAmWExv
Z0N0bC0+SW5zZXJ0OworCXVpbnQ2NAkJY3VycmVudF9ieXRlcG9zOworCisJU3BpbkxvY2tBY3F1
aXJlKCZJbnNlcnQtPmluc2VydHBvc19sY2spOworCWN1cnJlbnRfYnl0ZXBvcyA9IEluc2VydC0+
Q3VyckJ5dGVQb3M7CisJU3BpbkxvY2tSZWxlYXNlKCZJbnNlcnQtPmluc2VydHBvc19sY2spOwor
CisJcmV0dXJuIFhMb2dCeXRlUG9zVG9FbmRSZWNQdHIoY3VycmVudF9ieXRlcG9zKTsKK30KKwog
LyoKICAqIEdldCBsYXRlc3QgV0FMIHdyaXRlIHBvaW50ZXIKICAqLwpkaWZmIC0tZ2l0IGEvc3Jj
L2JhY2tlbmQvcmVwbGljYXRpb24vbG9naWNhbC9zeW5jdXRpbHMuYyBiL3NyYy9iYWNrZW5kL3Jl
cGxpY2F0aW9uL2xvZ2ljYWwvc3luY3V0aWxzLmMKaW5kZXggZWY2MWNhMDQzN2QuLjhjNWRhNDRk
NDJlIDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9yZXBsaWNhdGlvbi9sb2dpY2FsL3N5bmN1dGls
cy5jCisrKyBiL3NyYy9iYWNrZW5kL3JlcGxpY2F0aW9uL2xvZ2ljYWwvc3luY3V0aWxzLmMKQEAg
LTYyLDcgKzYyLDcgQEAgRmluaXNoU3luY1dvcmtlcih2b2lkKQogCX0KIAogCS8qIEFuZCBmbHVz
aCBhbGwgd3JpdGVzLiAqLwotCVhMb2dGbHVzaChHZXRYTG9nV3JpdGVSZWNQdHIoKSk7CisJWExv
Z0ZsdXNoKEdldFhMb2dJbnNlcnRFbmRSZWNQdHIoKSk7CiAKIAlpZiAoYW1fc2VxdWVuY2VzeW5j
X3dvcmtlcigpKQogCXsKZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3JlcGxpY2F0aW9uL3dhbHNl
bmRlci5jIGIvc3JjL2JhY2tlbmQvcmVwbGljYXRpb24vd2Fsc2VuZGVyLmMKaW5kZXggNzlmYzE5
MmIxNzEuLmRkNDZkZTdiY2Q2IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9yZXBsaWNhdGlvbi93
YWxzZW5kZXIuYworKysgYi9zcmMvYmFja2VuZC9yZXBsaWNhdGlvbi93YWxzZW5kZXIuYwpAQCAt
MTg4Nyw3ICsxODg3LDcgQEAgV2FsU25kV2FpdEZvcldhbChYTG9nUmVjUHRyIGxvYykKIAkJICog
d3JpdHRlbiwgYmVjYXVzZSB3YWx3cml0ZXIgaGFzIHNodXQgZG93biBhbHJlYWR5LgogCQkgKi8K
IAkJaWYgKGdvdF9TVE9QUElORyAmJiAhUmVjb3ZlcnlJblByb2dyZXNzKCkpCi0JCQlYTG9nRmx1
c2goR2V0WExvZ0luc2VydFJlY1B0cigpKTsKKwkJCVhMb2dGbHVzaChHZXRYTG9nSW5zZXJ0RW5k
UmVjUHRyKCkpOwogCiAJCS8qCiAJCSAqIFRvIGF2b2lkIHRoZSBzY2VuYXJpbyB3aGVyZSBzdGFu
ZGJ5cyBuZWVkIHRvIGNhdGNoIHVwIHRvIGEgbmV3ZXIKZGlmZiAtLWdpdCBhL3NyYy9pbmNsdWRl
L2FjY2Vzcy94bG9nLmggYi9zcmMvaW5jbHVkZS9hY2Nlc3MveGxvZy5oCmluZGV4IGZkZmI1NzI0
NjdiLi45NThmMzllZGRhNCAxMDA2NDQKLS0tIGEvc3JjL2luY2x1ZGUvYWNjZXNzL3hsb2cuaAor
KysgYi9zcmMvaW5jbHVkZS9hY2Nlc3MveGxvZy5oCkBAIC0yMzgsNiArMjM4LDcgQEAgZXh0ZXJu
IGJvb2wgUmVjb3ZlcnlJblByb2dyZXNzKHZvaWQpOwogZXh0ZXJuIFJlY292ZXJ5U3RhdGUgR2V0
UmVjb3ZlcnlTdGF0ZSh2b2lkKTsKIGV4dGVybiBib29sIFhMb2dJbnNlcnRBbGxvd2VkKHZvaWQp
OwogZXh0ZXJuIFhMb2dSZWNQdHIgR2V0WExvZ0luc2VydFJlY1B0cih2b2lkKTsKK2V4dGVybiBY
TG9nUmVjUHRyIEdldFhMb2dJbnNlcnRFbmRSZWNQdHIodm9pZCk7CiBleHRlcm4gWExvZ1JlY1B0
ciBHZXRYTG9nV3JpdGVSZWNQdHIodm9pZCk7CiAKIGV4dGVybiB1aW50NjQgR2V0U3lzdGVtSWRl
bnRpZmllcih2b2lkKTsKLS0gCjIuNTMuMAoK
--00000000000056230a064cd44820--