pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix 'unexpected data beyond EOF' on replica restart 6+ messages / 1 participants [nested] [flat]
* pgsql: Fix 'unexpected data beyond EOF' on replica restart @ 2026-01-15 19:11 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-01-15 19:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix 'unexpected data beyond EOF' on replica restart On restart, a replica can fail with an error like 'unexpected data beyond EOF in block 200 of relation T/D/R'. These are the steps to reproduce it: - A relation has a size of 400 blocks. - Blocks 201 to 400 are empty. - Block 200 has two rows. - Blocks 100 to 199 are empty. - A restartpoint is done - Vacuum truncates the relation to 200 blocks - A FPW deletes a row in block 200 - A checkpoint is done - A FPW deletes the last row in block 200 - Vacuum truncates the relation to 100 blocks - The replica restarts When the replica restarts: - The relation on disk starts at 100 blocks, because all the truncations were applied before restart. - The first truncate to 200 blocks is replayed. It silently fails, but it will still (incorrectly!) update the cache size to 200 blocks - The first FPW on block 200 is applied. XLogReadBufferForRead relies on the cached size and incorrectly assumes that the page already exists in the file, and thus won't extend the relation. - The online checkpoint record is replayed, calling smgrdestroyall which causes the cached size to be discarded - The second FPW on block 200 is applied. This time, the detected size is 100 blocks, an extend is attempted. However, the block 200 is already present in the buffer cache due to the first FPW. This triggers the 'unexpected data beyond EOF'. To fix, update the cached size in SmgrRelation with the current size rather than the requested new size, when the requested new size is greater. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Discussion: https://www.postgresql.org/message-id/CAO6_Xqrv-snNJNhbj1KjQmWiWHX3nYGDgAc=vxaZP3qc4g1Siw@mail.gmail... Backpatch-through: 14 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/23b25586dccfe81b6ac46ba8b4e505d48192732b Modified Files -------------- src/backend/storage/smgr/md.c | 3 +++ src/backend/storage/smgr/smgr.c | 11 ++++++++++- 2 files changed, 13 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix 'unexpected data beyond EOF' on replica restart @ 2026-01-15 19:11 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-01-15 19:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix 'unexpected data beyond EOF' on replica restart On restart, a replica can fail with an error like 'unexpected data beyond EOF in block 200 of relation T/D/R'. These are the steps to reproduce it: - A relation has a size of 400 blocks. - Blocks 201 to 400 are empty. - Block 200 has two rows. - Blocks 100 to 199 are empty. - A restartpoint is done - Vacuum truncates the relation to 200 blocks - A FPW deletes a row in block 200 - A checkpoint is done - A FPW deletes the last row in block 200 - Vacuum truncates the relation to 100 blocks - The replica restarts When the replica restarts: - The relation on disk starts at 100 blocks, because all the truncations were applied before restart. - The first truncate to 200 blocks is replayed. It silently fails, but it will still (incorrectly!) update the cache size to 200 blocks - The first FPW on block 200 is applied. XLogReadBufferForRead relies on the cached size and incorrectly assumes that the page already exists in the file, and thus won't extend the relation. - The online checkpoint record is replayed, calling smgrdestroyall which causes the cached size to be discarded - The second FPW on block 200 is applied. This time, the detected size is 100 blocks, an extend is attempted. However, the block 200 is already present in the buffer cache due to the first FPW. This triggers the 'unexpected data beyond EOF'. To fix, update the cached size in SmgrRelation with the current size rather than the requested new size, when the requested new size is greater. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Discussion: https://www.postgresql.org/message-id/CAO6_Xqrv-snNJNhbj1KjQmWiWHX3nYGDgAc=vxaZP3qc4g1Siw@mail.gmail... Backpatch-through: 14 Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/9ed411e084b7e25885d761a15e3a54818cf856a9 Modified Files -------------- src/backend/storage/smgr/md.c | 3 +++ src/backend/storage/smgr/smgr.c | 11 ++++++++++- 2 files changed, 13 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix 'unexpected data beyond EOF' on replica restart @ 2026-01-15 19:11 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-01-15 19:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix 'unexpected data beyond EOF' on replica restart On restart, a replica can fail with an error like 'unexpected data beyond EOF in block 200 of relation T/D/R'. These are the steps to reproduce it: - A relation has a size of 400 blocks. - Blocks 201 to 400 are empty. - Block 200 has two rows. - Blocks 100 to 199 are empty. - A restartpoint is done - Vacuum truncates the relation to 200 blocks - A FPW deletes a row in block 200 - A checkpoint is done - A FPW deletes the last row in block 200 - Vacuum truncates the relation to 100 blocks - The replica restarts When the replica restarts: - The relation on disk starts at 100 blocks, because all the truncations were applied before restart. - The first truncate to 200 blocks is replayed. It silently fails, but it will still (incorrectly!) update the cache size to 200 blocks - The first FPW on block 200 is applied. XLogReadBufferForRead relies on the cached size and incorrectly assumes that the page already exists in the file, and thus won't extend the relation. - The online checkpoint record is replayed, calling smgrdestroyall which causes the cached size to be discarded - The second FPW on block 200 is applied. This time, the detected size is 100 blocks, an extend is attempted. However, the block 200 is already present in the buffer cache due to the first FPW. This triggers the 'unexpected data beyond EOF'. To fix, update the cached size in SmgrRelation with the current size rather than the requested new size, when the requested new size is greater. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Discussion: https://www.postgresql.org/message-id/CAO6_Xqrv-snNJNhbj1KjQmWiWHX3nYGDgAc=vxaZP3qc4g1Siw@mail.gmail... Backpatch-through: 14 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/c3770181c83feeae546e3fb3d19f99f325fca8f4 Modified Files -------------- src/backend/storage/smgr/md.c | 3 +++ src/backend/storage/smgr/smgr.c | 17 +++++++++++++---- 2 files changed, 16 insertions(+), 4 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix 'unexpected data beyond EOF' on replica restart @ 2026-01-15 19:11 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-01-15 19:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix 'unexpected data beyond EOF' on replica restart On restart, a replica can fail with an error like 'unexpected data beyond EOF in block 200 of relation T/D/R'. These are the steps to reproduce it: - A relation has a size of 400 blocks. - Blocks 201 to 400 are empty. - Block 200 has two rows. - Blocks 100 to 199 are empty. - A restartpoint is done - Vacuum truncates the relation to 200 blocks - A FPW deletes a row in block 200 - A checkpoint is done - A FPW deletes the last row in block 200 - Vacuum truncates the relation to 100 blocks - The replica restarts When the replica restarts: - The relation on disk starts at 100 blocks, because all the truncations were applied before restart. - The first truncate to 200 blocks is replayed. It silently fails, but it will still (incorrectly!) update the cache size to 200 blocks - The first FPW on block 200 is applied. XLogReadBufferForRead relies on the cached size and incorrectly assumes that the page already exists in the file, and thus won't extend the relation. - The online checkpoint record is replayed, calling smgrdestroyall which causes the cached size to be discarded - The second FPW on block 200 is applied. This time, the detected size is 100 blocks, an extend is attempted. However, the block 200 is already present in the buffer cache due to the first FPW. This triggers the 'unexpected data beyond EOF'. To fix, update the cached size in SmgrRelation with the current size rather than the requested new size, when the requested new size is greater. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Discussion: https://www.postgresql.org/message-id/CAO6_Xqrv-snNJNhbj1KjQmWiWHX3nYGDgAc=vxaZP3qc4g1Siw@mail.gmail... Backpatch-through: 14 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a2eeb04f3a0fff86e0e94745003e705ec396d4ba Modified Files -------------- src/backend/storage/smgr/md.c | 3 +++ src/backend/storage/smgr/smgr.c | 17 +++++++++++++---- 2 files changed, 16 insertions(+), 4 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix 'unexpected data beyond EOF' on replica restart @ 2026-01-15 19:11 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-01-15 19:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix 'unexpected data beyond EOF' on replica restart On restart, a replica can fail with an error like 'unexpected data beyond EOF in block 200 of relation T/D/R'. These are the steps to reproduce it: - A relation has a size of 400 blocks. - Blocks 201 to 400 are empty. - Block 200 has two rows. - Blocks 100 to 199 are empty. - A restartpoint is done - Vacuum truncates the relation to 200 blocks - A FPW deletes a row in block 200 - A checkpoint is done - A FPW deletes the last row in block 200 - Vacuum truncates the relation to 100 blocks - The replica restarts When the replica restarts: - The relation on disk starts at 100 blocks, because all the truncations were applied before restart. - The first truncate to 200 blocks is replayed. It silently fails, but it will still (incorrectly!) update the cache size to 200 blocks - The first FPW on block 200 is applied. XLogReadBufferForRead relies on the cached size and incorrectly assumes that the page already exists in the file, and thus won't extend the relation. - The online checkpoint record is replayed, calling smgrdestroyall which causes the cached size to be discarded - The second FPW on block 200 is applied. This time, the detected size is 100 blocks, an extend is attempted. However, the block 200 is already present in the buffer cache due to the first FPW. This triggers the 'unexpected data beyond EOF'. To fix, update the cached size in SmgrRelation with the current size rather than the requested new size, when the requested new size is greater. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Discussion: https://www.postgresql.org/message-id/CAO6_Xqrv-snNJNhbj1KjQmWiWHX3nYGDgAc=vxaZP3qc4g1Siw@mail.gmail... Backpatch-through: 14 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/ef8465588c988aee7555df4a70ec4e584b9c9fe1 Modified Files -------------- src/backend/storage/smgr/md.c | 3 +++ src/backend/storage/smgr/smgr.c | 17 +++++++++++++---- 2 files changed, 16 insertions(+), 4 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix 'unexpected data beyond EOF' on replica restart @ 2026-01-15 19:11 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-01-15 19:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix 'unexpected data beyond EOF' on replica restart On restart, a replica can fail with an error like 'unexpected data beyond EOF in block 200 of relation T/D/R'. These are the steps to reproduce it: - A relation has a size of 400 blocks. - Blocks 201 to 400 are empty. - Block 200 has two rows. - Blocks 100 to 199 are empty. - A restartpoint is done - Vacuum truncates the relation to 200 blocks - A FPW deletes a row in block 200 - A checkpoint is done - A FPW deletes the last row in block 200 - Vacuum truncates the relation to 100 blocks - The replica restarts When the replica restarts: - The relation on disk starts at 100 blocks, because all the truncations were applied before restart. - The first truncate to 200 blocks is replayed. It silently fails, but it will still (incorrectly!) update the cache size to 200 blocks - The first FPW on block 200 is applied. XLogReadBufferForRead relies on the cached size and incorrectly assumes that the page already exists in the file, and thus won't extend the relation. - The online checkpoint record is replayed, calling smgrdestroyall which causes the cached size to be discarded - The second FPW on block 200 is applied. This time, the detected size is 100 blocks, an extend is attempted. However, the block 200 is already present in the buffer cache due to the first FPW. This triggers the 'unexpected data beyond EOF'. To fix, update the cached size in SmgrRelation with the current size rather than the requested new size, when the requested new size is greater. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Discussion: https://www.postgresql.org/message-id/CAO6_Xqrv-snNJNhbj1KjQmWiWHX3nYGDgAc=vxaZP3qc4g1Siw@mail.gmail... Backpatch-through: 14 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2514f1c774c075926a5d678d0091e1293cdb9f6d Modified Files -------------- src/backend/storage/smgr/md.c | 3 +++ src/backend/storage/smgr/smgr.c | 17 +++++++++++++---- 2 files changed, 16 insertions(+), 4 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2026-01-15 19:11 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-01-15 19:11 pgsql: Fix 'unexpected data beyond EOF' on replica restart Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-01-15 19:11 pgsql: Fix 'unexpected data beyond EOF' on replica restart Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-01-15 19:11 pgsql: Fix 'unexpected data beyond EOF' on replica restart Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-01-15 19:11 pgsql: Fix 'unexpected data beyond EOF' on replica restart Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-01-15 19:11 pgsql: Fix 'unexpected data beyond EOF' on replica restart Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-01-15 19:11 pgsql: Fix 'unexpected data beyond EOF' on replica restart Heikki Linnakangas <heikki.linnakangas@iki.fi>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox