MIME-Version: 1.0
References: 
 <CAHGQGwHFKF+x4E+SqedMCnmLCitxjTUUtSyL_+mMeuq-GbEt6w@mail.gmail.com>
 <CAO6_Xqp+ADb6KZVWLMALu3xmwVUEO8S1EiCnp38mG6BrHrEnuA@mail.gmail.com>
 <CAO6_XqqKDV+AuP=Gf4kRKPqzyYTsOyGd3LE8Jqkwi7EMPJpbhA@mail.gmail.com>
 <CAHGQGwHc5yH4Nxp59KXJP0kAr61j3W7QeSKT2HxVjZa3OrLzmg@mail.gmail.com>
 <CAO6_Xqq1h6kggb1o206rgouPS0H5jnjahzZ0We-9ggnBjB2JsA@mail.gmail.com>
 <CAHGQGwFJnNUOMiW9wR-2WjSKzzj0wV8p55J8bnJ6mik=z0oFPQ@mail.gmail.com>
 <2631a3c3-5e60-4a1a-9e20-377024322602@gmail.com>
 <CAO6_Xqq73TPa3M6nQ7RqRhKkcphy1JX7aNGTYy-x_Sn+6a8Z_Q@mail.gmail.com>
 <CAHGQGwGvnpN=2bo+F7H90YLFcx9=SazwLkcx+0gEcrbQy5NVZg@mail.gmail.com>
 <CAHGQGwECpyJtMqkCEvyqgZDiwAeMj3RKobui7jONrDd35W0x3Q@mail.gmail.com>
 <vzguaguldbcyfbyuq76qj7hx5qdr5kmh67gqkncyb2yhsygrdt@dfhcpteqifux>
 <CAO6_Xqq=-KnxrFQvmyF++XH4ngtXVhcDB979rEn6SPtUhSmNYg@mail.gmail.com>
In-Reply-To: 
 <CAO6_Xqq=-KnxrFQvmyF++XH4ngtXVhcDB979rEn6SPtUhSmNYg@mail.gmail.com>
From: Fujii Masao <masao.fujii@gmail.com>
Date: Fri, 13 Mar 2026 02:24:36 +0900
Message-ID: 
 <CAHGQGwGpfEsBJBLQDou7xccnb0tj1T_NQv2gZfuQfgzn_b=RsQ@mail.gmail.com>
Subject: Re: Shutdown indefinitely stuck due to unflushed FPI_FOR_HINT record
To: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Cc: Andres Freund <andres@anarazel.de>,
 Alexander Lakhin <exclusion@gmail.com>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CAHGQGwGpfEsBJBLQDou7xccnb0tj1T_NQv2gZfuQfgzn_b%3DRsQ%40mail.gmail.com>
Precedence: bulk

On Thu, Mar 12, 2026 at 11:08=E2=80=AFPM Anthonin Bonnefoy
<anthonin.bonnefoy@datadoghq.com> wrote:
>
> On Tue, Mar 10, 2026 at 6:11=E2=80=AFPM Andres Freund <andres@anarazel.de=
> wrote:
> > I'm pretty sure this is not correct as-is, it suffers from the same iss=
ue as
> > https://postgr.es/m/vf4hbwrotvhbgcnknrqmfbqlu75oyjkmausvy66ic7x7vuhafx%=
40e4rvwavtjswo
> > I.e. it is not safe to use GetXLogInsertRecPtr() to determine up to whe=
re to
> > flush to, due to page boundaries.

Thanks for the report!


> I've managed to reproduce this issue by ensuring the FPI_FOR_HINT
> record finishes at the end of a page with the following script (might
> need some adjustment if the record sizes are different):
>
> DROP TABLE IF EXISTS test_insert_rec_ptr;
> CREATE TABLE test_insert_rec_ptr(aid int, data text) WITH
> (autovacuum_enabled =3D false);
> INSERT INTO test_insert_rec_ptr SELECT *, repeat('a', 100) FROM
> generate_series(0, 57);
> -- This should tag the page as full
> BEGIN; UPDATE test_insert_rec_ptr SET aid=3D2 where aid=3D1; ROLLBACK;
> CHECKPOINT;
> -- Start with a fresh file
> SELECT pg_switch_wal();
> -- Our FPI_FOR_HINT writes 8193 bytes
> -- With the long header,  the first  page has 8152 bytes available
> -- With the short header, the second page has 8168 bytes available
> -- We want our FPI_FOR_HINT to finish at the end of the second page
> (+/- 8 bytes of alignment)
> -- We need to write the first 25 bytes (or 32 with alignment) in the firs=
t page
> -- For that, we need to write 8120 bytes of WAL records
> BEGIN;
> -- 264 bytes of FPW
> INSERT INTO test_insert_rec_ptr VALUES(1);
> -- 74 * 104 bytes
> INSERT INTO test_insert_rec_ptr SELECT *, repeat('a', 44) FROM
> generate_series(1, 74);
> -- 108 bytes
> INSERT INTO test_insert_rec_ptr VALUES(1, repeat('a', 48));
> -- 46 bytes
> COMMIT;
> -- 264 + 74 * 104 + 46 + 108 =3D 8114 bytes, which will round up to 8120
> with alignment
> -- FPI_FOR_HINT record should be at 0x1FE0
> BEGIN; SELECT * FROM test_insert_rec_ptr WHERE aid=3D2; ROLLBACK;
>
> As far as I can tell, the only impact it has is to complain about the
> write request being too far:
> LOG:  request to flush past end of generated WAL; request 0/01604018,
> current position 0/01604000
> ERROR:  xlog flush request 0/01604018 is not satisfied --- flushed
> only to 0/01604000
>
> To avoid this issue, it sounds like we need something to use
> XLogBytePosToEndRecPtr instead of XLogBytePosToRecPtr to convert the
> byte position? With XLogBytePosToRecPtr(), the flush request would
> stop at 01604000 instead of going to the next page with 01604018.
>
> In the attached patch, I've added a GetXLogInsertEndRecPtr() function
> which is similar to GetXLogInsertRecPtr(), except it uses
> XLogBytePosToEndRecPtr() to stop at the page boundary.
> There was also another XLogFlush(GetXLogWriteRecPtr()) call in
> syncutils.c, so I replaced both calls with
> XLogFlush(GetXLogInsertEndRecPtr()).

Thanks for investigating the issue and making the patch!
It looks good to me.

Andres,

Do you have any comments on the proposed patch?

Regards,

--=20
Fujii Masao