pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
7+ messages / 4 participants
[nested] [flat]
* BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
@ 2026-11-07 01:23 PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 7+ messages in thread
From: PG Bug reporting form @ 2026-11-07 01:23 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: 125atishprak@gmail.com
The following bug has been logged on the website:
Bug reference: 19839
Logged by: Atish Prakash
Email address: 125atishprak@gmail.com
PostgreSQL version: 18.6
Operating system: Linux x86_64 (kernel 7.0.0, glibc 2.43)
Description:
On an assert-enabled build, logical decoding can fail this assertion and
take
the server through crash recovery:
TRAP: failed Assert("XLogRecPtrIsValid(txn->final_lsn)"), File:
"reorderbuffer.c", Line: 5001
Expected: decoding discards the aborted transaction and continues.
The sequence is:
1. A transaction has a subtransaction with an XID but no decoded changes,
for
example one that only ran SELECT ... FOR UPDATE.
2. The parent exceeds logical_decoding_work_mem while still in progress.
ReorderBufferSerializeTXN() recurses into the subtransaction, writes
nothing,
and still sets RBTXN_IS_SERIALIZED on it. Its final_lsn stays invalid.
3. The parent aborts while the decoder is behind.
4. The parent is chosen for eviction before its abort record is decoded.
ReorderBufferCheckAndTruncateAbortedTXN() finds it aborted and truncates
it,
and ReorderBufferRestoreCleanup() fails on the subtransaction's
final_lsn.
ReorderBufferCheckAndTruncateAbortedTXN() came in with 072ee847ad, so I
expect
17 and earlier are not affected, but I have not tested them.
From reading the code, a non-assert build goes through an empty cleanup loop
for
this subtransaction, so I expect no user-visible effect there. I have not
run a
non-assert build.
I found this with a stress test of REPACK (CONCURRENTLY), where the REPACK
decoding worker failed in the same way, on both a plain assert build and an
ASan
build. It does not need REPACK: the attached script uses only
pg_recvlogical. It
stalls pg_recvlogical with SIGSTOP so that the decoder is behind when the
transaction aborts.
Proposed fix, attached as 0001-fix-aborted-spilled-txn-assert.patch: set
RBTXN_IS_SERIALIZED only if something was written. With the patch, on
master:
- the script no longer fails
- all test_decoding tests and all 239 core regression tests pass
- a spilled transaction with an empty subtransaction, a rolled-back
subtransaction
and a released one decodes the expected rows
TAP tests were not run, because my build has them disabled. The patch has no
regression test, because the script depends on stalling a live consumer. It
applies cleanly to REL_18_STABLE and REL_19_STABLE, but I only built and
tested it
on master.
I found this with a fuzzer, and AI tools helped me build it, write the
script
and patch, and draft this report.
Versions where the script fails:
- 18.6 (REL_18_STABLE at 813a74bc5a; the assertion reads txn->final_lsn !=
InvalidXLogRecPtr)
- 19rc1 (REL_19_STABLE at 0b1eb8b917)
- master at 72b9498df9, plus my unrelated uncommitted patch to ruleutils.c
The patch was built on master at b58a9a97fe.
Platform: Linux 7.0.0-42-generic x86_64, glibc 2.43, Intel i9-13900HX, 14
GiB RAM
Build: clang 21.1.8, CFLAGS=-O1 -g, --enable-cassert --enable-debug
--without-icu --without-readline --without-libxml --without-openssl
Settings set by the script: wal_level=logical,
logical_decoding_work_mem=64kB
Stack from 19rc1, with addr2line for the static functions:
ReorderBufferRestoreCleanup reorderbuffer.c:5001
ReorderBufferTruncateTXN reorderbuffer.c:1749
(subtransaction)
ReorderBufferTruncateTXN reorderbuffer.c:1665 (parent)
ReorderBufferCheckAndTruncateAbortedTXN reorderbuffer.c:1819
ReorderBufferQueueChange
DecodeInsert decode.c:956
LogicalDecodingProcessRecord
Reproducer, run as: logical-decoding-assert-repro.sh <bin dir of an assert
build>
#!/bin/bash
# Usage: logical-decoding-assert-repro.sh <postgres bin dir, built with
--enable-cassert> [work dir]
# Assertion failure in ReorderBufferRestoreCleanup during logical decoding,
without REPACK.
set -u
BIN=$1; WORK=${2:-$(mktemp -d)}; PORT=${PORT:-6399}; D=$WORK/data; rm -rf
"$D"; mkdir -p "$WORK"
export PGHOST=$WORK PGPORT=$PORT PGDATABASE=postgres
"$BIN/initdb" -D "$D" -A trust --no-sync >/dev/null
cat >> "$D/postgresql.conf" <<CONF
listen_addresses = ''
unix_socket_directories = '$WORK'
port = $PORT
wal_level = logical
logical_decoding_work_mem = 64kB
CONF
"$BIN/pg_ctl" -D "$D" -l "$WORK/server.log" -w start >/dev/null
P="$BIN/psql -X -q -v ON_ERROR_STOP=1"
$P -c "CREATE TABLE t (a text)" -c "INSERT INTO t VALUES ('locked')" \
-c "SELECT pg_create_logical_replication_slot('s', 'test_decoding')"
>/dev/null
"$BIN/pg_recvlogical" -d postgres -S s --start -f /dev/null -F 0 -s 1 &
RECV=$!
# Session A stays open throughout.
FIFO=$WORK/a.fifo; rm -f "$FIFO"; mkfifo "$FIFO"; $P < "$FIFO" > /dev/null &
APSQL=$!; exec 9> "$FIFO"
# A: a subtransaction with an XID but no decoded change, then enough to
exceed the limit.
cat >&9 <<SQL
BEGIN;
INSERT INTO t VALUES ('first');
SAVEPOINT sp;
SELECT * FROM t WHERE a = 'locked' FOR UPDATE;
RELEASE sp;
INSERT INTO t SELECT repeat('a', 100) FROM generate_series(1, 2000);
SELECT 1;
SQL
sleep 2 # the walsender decodes this and spills A; A is still in progress
# Stall the walsender in the middle of sending a large committed
transaction.
kill -STOP $RECV
$P -c "INSERT INTO t SELECT repeat('c', 1000) FROM generate_series(1,
200000)"
for i in $(seq 1 120); do
w=$($P -Atc "SELECT wait_event FROM pg_stat_activity WHERE backend_type =
'walsender'")
[ "$w" = "WalSenderWriteData" ] && break
sleep 0.5
done
echo "walsender wait event: $w"
# Behind the stalled walsender: more of A, then another transaction that
crosses the limit, then A aborts.
echo "INSERT INTO t SELECT repeat('a', 100) FROM generate_series(1, 200);
SELECT 1;" >&9
sleep 0.5
$P <<SQL &
BEGIN;
INSERT INTO t SELECT repeat('d', 100) FROM generate_series(1, 120);
SELECT pg_sleep(4);
COMMIT;
SQL
DS=$!
sleep 1
echo "ROLLBACK;" >&9
exec 9>&-
wait $APSQL
kill -CONT $RECV # decoding resumes; A's abort record has not been read
but CLOG says aborted
sleep 8
wait $DS 2>/dev/null
kill $RECV 2>/dev/null
echo "--- server log"
grep -E 'TRAP|terminated by signal|was terminated' "$WORK/server.log" | cut
-c1-200
"$BIN/pg_ctl" -D "$D" -m immediate stop >/dev/null 2>&1
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
@ 2026-11-07 01:26 Atisss hh <125atishprak@gmail.com>
parent: PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 7+ messages in thread
From: Atisss hh @ 2026-11-07 01:26 UTC (permalink / raw)
To: 125atishprak@gmail.com; pgsql-bugs@lists.postgresql.org
On Sat, 7 Nov 2026 at 06:54, PG Bug reporting form <noreply@postgresql.org>
wrote:
> The following bug has been logged on the website:
>
> Bug reference: 19839
> Logged by: Atish Prakash
> Email address: 125atishprak@gmail.com
> PostgreSQL version: 18.6
> Operating system: Linux x86_64 (kernel 7.0.0, glibc 2.43)
> Description:
>
> On an assert-enabled build, logical decoding can fail this assertion and
> take
> the server through crash recovery:
>
> TRAP: failed Assert("XLogRecPtrIsValid(txn->final_lsn)"), File:
> "reorderbuffer.c", Line: 5001
>
> Expected: decoding discards the aborted transaction and continues.
>
> The sequence is:
>
> 1. A transaction has a subtransaction with an XID but no decoded changes,
> for
> example one that only ran SELECT ... FOR UPDATE.
> 2. The parent exceeds logical_decoding_work_mem while still in progress.
> ReorderBufferSerializeTXN() recurses into the subtransaction, writes
> nothing,
> and still sets RBTXN_IS_SERIALIZED on it. Its final_lsn stays invalid.
> 3. The parent aborts while the decoder is behind.
> 4. The parent is chosen for eviction before its abort record is decoded.
> ReorderBufferCheckAndTruncateAbortedTXN() finds it aborted and truncates
> it,
> and ReorderBufferRestoreCleanup() fails on the subtransaction's
> final_lsn.
>
> ReorderBufferCheckAndTruncateAbortedTXN() came in with 072ee847ad, so I
> expect
> 17 and earlier are not affected, but I have not tested them.
>
> From reading the code, a non-assert build goes through an empty cleanup
> loop
> for
> this subtransaction, so I expect no user-visible effect there. I have not
> run a
> non-assert build.
>
> I found this with a stress test of REPACK (CONCURRENTLY), where the REPACK
> decoding worker failed in the same way, on both a plain assert build and an
> ASan
> build. It does not need REPACK: the attached script uses only
> pg_recvlogical. It
> stalls pg_recvlogical with SIGSTOP so that the decoder is behind when the
> transaction aborts.
>
> Proposed fix, attached as 0001-fix-aborted-spilled-txn-assert.patch: set
> RBTXN_IS_SERIALIZED only if something was written. With the patch, on
> master:
> - the script no longer fails
> - all test_decoding tests and all 239 core regression tests pass
> - a spilled transaction with an empty subtransaction, a rolled-back
> subtransaction
> and a released one decodes the expected rows
> TAP tests were not run, because my build has them disabled. The patch has
> no
> regression test, because the script depends on stalling a live consumer. It
> applies cleanly to REL_18_STABLE and REL_19_STABLE, but I only built and
> tested it
> on master.
>
> I found this with a fuzzer, and AI tools helped me build it, write the
> script
> and patch, and draft this report.
>
> Versions where the script fails:
> - 18.6 (REL_18_STABLE at 813a74bc5a; the assertion reads txn->final_lsn !=
> InvalidXLogRecPtr)
> - 19rc1 (REL_19_STABLE at 0b1eb8b917)
> - master at 72b9498df9, plus my unrelated uncommitted patch to ruleutils.c
> The patch was built on master at b58a9a97fe.
> Platform: Linux 7.0.0-42-generic x86_64, glibc 2.43, Intel i9-13900HX, 14
> GiB RAM
> Build: clang 21.1.8, CFLAGS=-O1 -g, --enable-cassert --enable-debug
> --without-icu --without-readline --without-libxml --without-openssl
> Settings set by the script: wal_level=logical,
> logical_decoding_work_mem=64kB
>
> Stack from 19rc1, with addr2line for the static functions:
>
> ReorderBufferRestoreCleanup reorderbuffer.c:5001
> ReorderBufferTruncateTXN reorderbuffer.c:1749
> (subtransaction)
> ReorderBufferTruncateTXN reorderbuffer.c:1665 (parent)
> ReorderBufferCheckAndTruncateAbortedTXN reorderbuffer.c:1819
> ReorderBufferQueueChange
> DecodeInsert decode.c:956
> LogicalDecodingProcessRecord
>
> Reproducer, run as: logical-decoding-assert-repro.sh <bin dir of an assert
> build>
>
> #!/bin/bash
> # Usage: logical-decoding-assert-repro.sh <postgres bin dir, built with
> --enable-cassert> [work dir]
> # Assertion failure in ReorderBufferRestoreCleanup during logical decoding,
> without REPACK.
> set -u
> BIN=$1; WORK=${2:-$(mktemp -d)}; PORT=${PORT:-6399}; D=$WORK/data; rm -rf
> "$D"; mkdir -p "$WORK"
> export PGHOST=$WORK PGPORT=$PORT PGDATABASE=postgres
> "$BIN/initdb" -D "$D" -A trust --no-sync >/dev/null
> cat >> "$D/postgresql.conf" <<CONF
> listen_addresses = ''
> unix_socket_directories = '$WORK'
> port = $PORT
> wal_level = logical
> logical_decoding_work_mem = 64kB
> CONF
> "$BIN/pg_ctl" -D "$D" -l "$WORK/server.log" -w start >/dev/null
> P="$BIN/psql -X -q -v ON_ERROR_STOP=1"
> $P -c "CREATE TABLE t (a text)" -c "INSERT INTO t VALUES ('locked')" \
> -c "SELECT pg_create_logical_replication_slot('s', 'test_decoding')"
> >/dev/null
>
> "$BIN/pg_recvlogical" -d postgres -S s --start -f /dev/null -F 0 -s 1 &
> RECV=$!
>
> # Session A stays open throughout.
> FIFO=$WORK/a.fifo; rm -f "$FIFO"; mkfifo "$FIFO"; $P < "$FIFO" > /dev/null
> &
> APSQL=$!; exec 9> "$FIFO"
> # A: a subtransaction with an XID but no decoded change, then enough to
> exceed the limit.
> cat >&9 <<SQL
> BEGIN;
> INSERT INTO t VALUES ('first');
> SAVEPOINT sp;
> SELECT * FROM t WHERE a = 'locked' FOR UPDATE;
> RELEASE sp;
> INSERT INTO t SELECT repeat('a', 100) FROM generate_series(1, 2000);
> SELECT 1;
> SQL
> sleep 2 # the walsender decodes this and spills A; A is still in
> progress
>
> # Stall the walsender in the middle of sending a large committed
> transaction.
> kill -STOP $RECV
> $P -c "INSERT INTO t SELECT repeat('c', 1000) FROM generate_series(1,
> 200000)"
> for i in $(seq 1 120); do
> w=$($P -Atc "SELECT wait_event FROM pg_stat_activity WHERE backend_type =
> 'walsender'")
> [ "$w" = "WalSenderWriteData" ] && break
> sleep 0.5
> done
> echo "walsender wait event: $w"
>
> # Behind the stalled walsender: more of A, then another transaction that
> crosses the limit, then A aborts.
> echo "INSERT INTO t SELECT repeat('a', 100) FROM generate_series(1, 200);
> SELECT 1;" >&9
> sleep 0.5
> $P <<SQL &
> BEGIN;
> INSERT INTO t SELECT repeat('d', 100) FROM generate_series(1, 120);
> SELECT pg_sleep(4);
> COMMIT;
> SQL
> DS=$!
> sleep 1
> echo "ROLLBACK;" >&9
> exec 9>&-
> wait $APSQL
> kill -CONT $RECV # decoding resumes; A's abort record has not been read
> but CLOG says aborted
> sleep 8
> wait $DS 2>/dev/null
> kill $RECV 2>/dev/null
> echo "--- server log"
> grep -E 'TRAP|terminated by signal|was terminated' "$WORK/server.log" | cut
> -c1-200
> "$BIN/pg_ctl" -D "$D" -m immediate stop >/dev/null 2>&1
>
>
>
Attachments:
0001-fix-aborted-spilled-txn-assert.patch (text/x-patch, 1.9K) download
From 77ffb554871482a1372bb16e7a76f15410f03e23 Mon Sep 17 00:00:00 2001
From: Atish <125atishprak@gmail.com>
Date: Sat, 7 Nov 2026 01:52:24 +0530
Subject: [PATCH] Fix assertion failure when truncating an aborted spilled
transaction
ReorderBufferSerializeTXN() marked a transaction as serialized even when
it had no changes to write. This happens to a subtransaction that has an
XID but no decoded changes, such as one that only locked a row, when its
parent is spilled to disk. Such a subtransaction has no valid final_lsn
until its commit or abort is decoded.
If the parent is later evicted again and is found to have aborted before
its abort record is decoded, ReorderBufferCheckAndTruncateAbortedTXN()
truncates it and ReorderBufferRestoreCleanup() fails its assertion on the
subtransaction's final_lsn.
Mark the transaction as serialized only if something was written.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
src/backend/replication/logical/reorderbuffer.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 4684398531..2b9624e6e0 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -4216,7 +4216,16 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
+
+ /*
+ * Mark the transaction as serialized only if we wrote something. A
+ * subtransaction without any decoded changes gets here too, and it has no
+ * valid final_lsn until its commit or abort is decoded, which
+ * ReorderBufferRestoreCleanup() would need if the transaction were found
+ * to be aborted before that.
+ */
+ if (spilled)
+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
--
2.53.0
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
@ 2026-11-08 15:11 Zhijie Hou <houzhijie22@gmail.com>
parent: Atisss hh <125atishprak@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Zhijie Hou @ 2026-11-08 15:11 UTC (permalink / raw)
To: Atisss hh <125atishprak@gmail.com>; pgsql-bugs@lists.postgresql.org
Hi,
On Sat, Nov 7, 2026 at 09:26 AM Atisss hh <125atishprak@gmail.com>wrote:
>
>
> On Sat, 7 Nov 2026 at 06:54, PG Bug reporting form
> <noreply@postgresql.org <mailto:noreply@postgresql.org>> wrote:
>
> The following bug has been logged on the website:
>
> Bug reference: 19839
> Logged by: Atish Prakash
> Email address: 125atishprak@gmail.com <mailto:125atishprak@gmail.com>
> PostgreSQL version: 18.6
> Operating system: Linux x86_64 (kernel 7.0.0, glibc 2.43)
> Description:
>
> On an assert-enabled build, logical decoding can fail this assertion and
> take
> the server through crash recovery:
>
> TRAP: failed Assert("XLogRecPtrIsValid(txn->final_lsn)"), File:
> "reorderbuffer.c", Line: 5001
>
> Expected: decoding discards the aborted transaction and continues.
>
> The sequence is:
>
> 1. A transaction has a subtransaction with an XID but no decoded
> changes,
> for
> example one that only ran SELECT ... FOR UPDATE.
> 2. The parent exceeds logical_decoding_work_mem while still in progress.
> ReorderBufferSerializeTXN() recurses into the subtransaction, writes
> nothing,
> and still sets RBTXN_IS_SERIALIZED on it. Its final_lsn stays
> invalid.
> 3. The parent aborts while the decoder is behind.
> 4. The parent is chosen for eviction before its abort record is decoded.
> ReorderBufferCheckAndTruncateAbortedTXN() finds it aborted and
> truncates
> it,
> and ReorderBufferRestoreCleanup() fails on the subtransaction's
> final_lsn.
>
> ReorderBufferCheckAndTruncateAbortedTXN() came in with 072ee847ad, so I
> expect
> 17 and earlier are not affected, but I have not tested them.
Thanks for the report and patch.
I think this issue affects can all supported branches (back to PG 14),
and does not require an aborted transaction to reproduce.
For example, if a top-level transaction has a subtransaction with an XID
but no decoded changes, and then spills to disk while queuing a partial
change (such as TOAST chunks), the empty subtransaction is marked
RBTXN_IS_SERIALIZED with final_lsn == InvalidXLogRecPtr. Once the main
table insert completes the partial change,
ReorderBufferProcessPartialChange() streams the transaction via
ReorderBufferStreamTXN(), and ReorderBufferTruncateTXN() ->
ReorderBufferRestoreCleanup() hits the same assertion on the empty
subtransaction.
See the attachment for a test to reproduce this issue. (Feel free to
merge if it's worth adding a test to cover this path)
>
> From reading the code, a non-assert build goes through an empty
> cleanup loop
> for
> this subtransaction, so I expect no user-visible effect there. I
> have not
> run a
> non-assert build.
I think in a rare case, this can also affect the stats collection for
spilled txns. For example, if an empty subtransaction is marked as
serialized, we skip incrementing spillTxns because no changes were
written; if real changes are later serialized in this subtransaction, we
still do not increment spillTxns because the subtransaction is already
marked as serialized and we avoid counting it "twice".
>
> Proposed fix, attached as 0001-fix-aborted-spilled-txn-assert.patch: set
> RBTXN_IS_SERIALIZED only if something was written.
And I have one minor comment for the patch:
+ if (spilled)
+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
ReorderBufferSerializeTXN() already has an "if (spilled)" block right
above this, so we shall directly move the assignment into that block.
--
Best Regards,
Zhijie Hou
Attachments:
nocfbot-0001-Top-up-for-0001-Add-regression-test-for-spilled-txn-.patch (text/plain, 2.8K) download
From d57466124f66ed276b7ab1f009fa659bd9987454 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzhijie22@gmail.com>
Date: Sun, 8 Nov 2026 22:44:19 +0800
Subject: [PATCH] Top-up for 0001: Add regression test for spilled txn with
empty subxact
Author: Zhijie Hou <houzhijie22@gmail.com>
---
contrib/test_decoding/expected/stream.out | 24 +++++++++++++++++++++++
contrib/test_decoding/sql/stream.sql | 15 ++++++++++++++
2 files changed, 39 insertions(+)
diff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.out
index 0ec5c933610..9c164847b03 100644
--- a/contrib/test_decoding/expected/stream.out
+++ b/contrib/test_decoding/expected/stream.out
@@ -157,6 +157,30 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
COMMIT
(3 rows)
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change
+-- (stream_test.data has EXTERNAL storage set above). Otherwise, truncating
+-- the transaction after streaming fails on the subtransaction's invalid
+-- final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+ ?column?
+----------
+ 1
+(1 row)
+
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+ count
+-------
+ 4
+(1 row)
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sql
index 5e45a8e9b64..f3159335564 100644
--- a/contrib/test_decoding/sql/stream.sql
+++ b/contrib/test_decoding/sql/stream.sql
@@ -82,5 +82,20 @@ ROLLBACK;
INSERT INTO stream_test VALUES ('after-abort');
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change
+-- (stream_test.data has EXTERNAL storage set above). Otherwise, truncating
+-- the transaction after streaming fails on the subtransaction's invalid
+-- final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
--
2.54.0 (Apple Git-157)
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
@ 2026-11-09 10:53 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
parent: Zhijie Hou <houzhijie22@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Bertrand Drouvot @ 2026-11-09 10:53 UTC (permalink / raw)
To: Zhijie Hou <houzhijie22@gmail.com>; +Cc: Atisss hh <125atishprak@gmail.com>; pgsql-bugs@lists.postgresql.org
Hi,
On Sun, Nov 08, 2026 at 11:11:09PM +0800, Zhijie Hou wrote:
> Thanks for the report and patch.
+1.
> See the attachment for a test to reproduce this issue. (Feel free to merge
> if it's worth adding a test to cover this path)
> I think in a rare case, this can also affect the stats collection for
> spilled txns. For example, if an empty subtransaction is marked as
> serialized, we skip incrementing spillTxns because no changes were written;
> if real changes are later serialized in this subtransaction, we still do not
> increment spillTxns because the subtransaction is already marked as
> serialized and we avoid counting it "twice".
Agreed.
That's why I think the test is worth adding (as the bug could also produce real
user visible impact as it can cause spill_txns to be undercounted on non assert
enabled builds). The test only detects the regression on assert enabled builds,
but I think that is enough. A test for the spill_txns case would be more
complicated and is probably not worth it.
>
> + if (spilled)
> + txn->txn_flags |= RBTXN_IS_SERIALIZED;
>
> ReorderBufferSerializeTXN() already has an "if (spilled)" block right above
> this, so we shall directly move the assignment into that block.
+1.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
@ 2026-11-09 11:46 Zhijie Hou <houzhijie22@gmail.com>
parent: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Zhijie Hou @ 2026-11-09 11:46 UTC (permalink / raw)
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>; +Cc: Atisss hh <125atishprak@gmail.com>; pgsql-bugs@lists.postgresql.org
Hi,
On Mon, Nov 9, 2026 at 06:53 PM Bertrand Drouvot wrote:
>
> On Sun, Nov 08, 2026 at 11:11:09PM +0800, Zhijie Hou wrote:
> > Thanks for the report and patch.
>
> +1.
>
> > See the attachment for a test to reproduce this issue. (Feel free to merge
> > if it's worth adding a test to cover this path)
>
> > I think in a rare case, this can also affect the stats collection
> > for spilled txns. For example, if an empty subtransaction is marked
> > as
> > serialized, we skip incrementing spillTxns because no changes were written;
> > if real changes are later serialized in this subtransaction, we still do not
> > increment spillTxns because the subtransaction is already marked as
> > serialized and we avoid counting it "twice".
>
> Agreed.
>
> That's why I think the test is worth adding (as the bug could also
> produce real user visible impact as it can cause spill_txns to be
> undercounted on non assert enabled builds). The test only detects the
> regression on assert enabled builds, but I think that is enough. A
> test for the spill_txns case would be more complicated and is probably
> not worth it.
>
> >
> > + if (spilled)
> > + txn->txn_flags |= RBTXN_IS_SERIALIZED;
> >
> > ReorderBufferSerializeTXN() already has an "if (spilled)" block
> > right above this, so we shall directly move the assignment into that
> > block.
>
> +1.
Thanks for the review. It looks like Atisss's update patches didn't
reach the list, probably a missing Reply-All, so I'm helping to attach
the same patches (provided by Atisss) here.
--
Best Regards,
Zhijie Hou
Attachments: download all (3 files, .tar.gz)
v2-0001-fix-empty-subxact-serialized-flag.patch (text/x-patch, 4.7K) download
From 092dc34678cd151de4262322bda1b2d31b1cec11 Mon Sep 17 00:00:00 2001
From: Atish Prakash <125atishprak@gmail.com>
Date: Sun, 8 Nov 2026 21:06:11 +0530
Subject: [PATCH v2] Don't mark a transaction serialized when nothing was
spilled
ReorderBufferSerializeTXN() set RBTXN_IS_SERIALIZED on every transaction
it visited, including subtransactions with no changes to write. Such a
transaction keeps an invalid final_lsn, because final_lsn is only advanced
as changes are written, so ReorderBufferRestoreCleanup() can later fail its
assertion on it. This happens when ReorderBufferCheckAndTruncateAbortedTXN()
truncates an aborted transaction, and when a transaction is truncated after
being streamed once a partial change completes.
It also made spill_txns undercount: when such a subtransaction later spilled
real changes, it was not counted, as it was already marked serialized.
Set the flag only when changes were written, after the statistics update
that checks it.
Test case by Zhijie Hou.
Bug: #19839
Reported-by: Atish Prakash <125atishprak@gmail.com>
Author: Atish Prakash <125atishprak@gmail.com>
Reviewed-by: Zhijie Hou <houzhijie22@gmail.com>
Backpatch-through: 14
---
contrib/test_decoding/expected/stream.out | 24 +++++++++++++++++++
contrib/test_decoding/sql/stream.sql | 15 ++++++++++++
.../replication/logical/reorderbuffer.c | 11 ++++++++-
3 files changed, 49 insertions(+), 1 deletion(-)
diff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.out
index 0ec5c93361..9c164847b0 100644
--- a/contrib/test_decoding/expected/stream.out
+++ b/contrib/test_decoding/expected/stream.out
@@ -157,6 +157,30 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
COMMIT
(3 rows)
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change
+-- (stream_test.data has EXTERNAL storage set above). Otherwise, truncating
+-- the transaction after streaming fails on the subtransaction's invalid
+-- final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+ ?column?
+----------
+ 1
+(1 row)
+
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+ count
+-------
+ 4
+(1 row)
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sql
index 5e45a8e9b6..f315933556 100644
--- a/contrib/test_decoding/sql/stream.sql
+++ b/contrib/test_decoding/sql/stream.sql
@@ -82,5 +82,20 @@ ROLLBACK;
INSERT INTO stream_test VALUES ('after-abort');
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change
+-- (stream_test.data has EXTERNAL storage set above). Otherwise, truncating
+-- the transaction after streaming fails on the subtransaction's invalid
+-- final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 4684398531..acbd2c1344 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -4211,12 +4211,21 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* update the decoding stats */
UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
+
+ /*
+ * Mark the transaction as serialized only if something was written,
+ * and only after the check above. final_lsn is advanced for each
+ * change written, so a transaction that wrote nothing, such as a
+ * subtransaction without decoded changes, would be marked with an
+ * invalid final_lsn, and ReorderBufferRestoreCleanup() would later
+ * fail on it.
+ */
+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
}
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
--
2.53.0
v2-REL_14_STABLE.patch (text/x-patch, 3.3K) download
diff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.out
index 0f21dcb8e0..0b6288fb14 100644
--- a/contrib/test_decoding/expected/stream.out
+++ b/contrib/test_decoding/expected/stream.out
@@ -106,6 +106,29 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL,NULL, 'incl
committing streamed transaction
(17 rows)
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change.
+-- Otherwise, truncating the transaction after streaming fails on the
+-- subtransaction's invalid final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+ ?column?
+----------
+ 1
+(1 row)
+
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+ count
+-------
+ 4
+(1 row)
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sql
index 4feec62972..1541e5dca3 100644
--- a/contrib/test_decoding/sql/stream.sql
+++ b/contrib/test_decoding/sql/stream.sql
@@ -44,5 +44,19 @@ toasted-123456789012345678901234567890123456789012345678901234567890123456789012
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL,NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change.
+-- Otherwise, truncating the transaction after streaming fails on the
+-- subtransaction's invalid final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 48a19ed4f6..71bf1e8e3d 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3896,12 +3896,21 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* update the decoding stats */
UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
+
+ /*
+ * Mark the transaction as serialized only if something was written,
+ * and only after the check above. final_lsn is advanced for each
+ * change written, so a transaction that wrote nothing, such as a
+ * subtransaction without decoded changes, would be marked with an
+ * invalid final_lsn, and ReorderBufferRestoreCleanup() would later
+ * fail on it.
+ */
+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
}
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
v2-REL_17_STABLE.patch (text/x-patch, 3.2K) download
diff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.out
index a76f77601e..a4b51c0051 100644
--- a/contrib/test_decoding/expected/stream.out
+++ b/contrib/test_decoding/expected/stream.out
@@ -128,6 +128,29 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL,
5
(1 row)
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change.
+-- Otherwise, truncating the transaction after streaming fails on the
+-- subtransaction's invalid final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+ ?column?
+----------
+ 1
+(1 row)
+
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+ count
+-------
+ 4
+(1 row)
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sql
index 7f43f0c2ab..22a4e1255b 100644
--- a/contrib/test_decoding/sql/stream.sql
+++ b/contrib/test_decoding/sql/stream.sql
@@ -59,5 +59,19 @@ ROLLBACK TO s1;
COMMIT;
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change.
+-- Otherwise, truncating the transaction after streaming fails on the
+-- subtransaction's invalid final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 50989db6c4..1eb42b1396 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -4049,12 +4049,21 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* update the decoding stats */
UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
+
+ /*
+ * Mark the transaction as serialized only if something was written,
+ * and only after the check above. final_lsn is advanced for each
+ * change written, so a transaction that wrote nothing, such as a
+ * subtransaction without decoded changes, would be marked with an
+ * invalid final_lsn, and ReorderBufferRestoreCleanup() would later
+ * fail on it.
+ */
+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
}
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
@ 2026-11-09 14:47 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
parent: Zhijie Hou <houzhijie22@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Bertrand Drouvot @ 2026-11-09 14:47 UTC (permalink / raw)
To: Zhijie Hou <houzhijie22@gmail.com>; +Cc: Atisss hh <125atishprak@gmail.com>; pgsql-bugs@lists.postgresql.org
Hi,
On Mon, Nov 09, 2026 at 07:46:59PM +0800, Zhijie Hou wrote:
> Thanks for the review. It looks like Atisss's update patches didn't reach
> the list, probably a missing Reply-All, so I'm helping to attach the same
> patches (provided by Atisss) here.
LGTM, just one nit about:
+ /*
+ * Mark the transaction as serialized only if something was written,
+ * and only after the check above. final_lsn is advanced for each
+ * change written, so a transaction that wrote nothing, such as a
+ * subtransaction without decoded changes, would be marked with an
+ * invalid final_lsn, and ReorderBufferRestoreCleanup() would later
+ * fail on it.
+ */
It looks lengthy and “the check above” is ambiguous. I'm not sure we need a new
comment here, maybe just replace the existing?
/* update the statistics iff we have spilled anything */
with
/* update the statistics and mark the transaction serialized iff we spilled */
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
@ 2026-11-12 15:40 Atisss hh <125atishprak@gmail.com>
parent: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 7+ messages in thread
From: Atisss hh @ 2026-11-12 15:40 UTC (permalink / raw)
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>; +Cc: Zhijie Hou <houzhijie22@gmail.com>; pgsql-bugs@lists.postgresql.org
Hi,
Really sorry for the late reply, I only saw your mail now.
Thanks for the review. v3 is attached. I dropped the long comment and
changed the existing one as you suggested:
/* update the statistics and mark the transaction serialized iff we
spilled */
The code is otherwise unchanged from v2. The flag is still set at the end
of the "if (spilled)" block, after the spillTxns line that checks it.
With v3, all test_decoding regression (20) and isolation (16) tests pass on
master. The 17 and 14 backports are updated the same way, in case they help.
I did not re-run their tests after the comment change, since only the
comment differs from v2.
Regards,
Atish Prakash
On Mon, 9 Nov 2026 at 20:17, Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
wrote:
> Hi,
>
> On Mon, Nov 09, 2026 at 07:46:59PM +0800, Zhijie Hou wrote:
> > Thanks for the review. It looks like Atisss's update patches didn't reach
> > the list, probably a missing Reply-All, so I'm helping to attach the same
> > patches (provided by Atisss) here.
>
> LGTM, just one nit about:
>
> + /*
> + * Mark the transaction as serialized only if something
> was written,
> + * and only after the check above. final_lsn is advanced
> for each
> + * change written, so a transaction that wrote nothing,
> such as a
> + * subtransaction without decoded changes, would be marked
> with an
> + * invalid final_lsn, and ReorderBufferRestoreCleanup()
> would later
> + * fail on it.
> + */
>
>
> It looks lengthy and “the check above” is ambiguous. I'm not sure we need
> a new
> comment here, maybe just replace the existing?
>
> /* update the statistics iff we have spilled anything */
>
> with
>
> /* update the statistics and mark the transaction serialized iff we
> spilled */
>
>
> Regards,
>
> --
> Bertrand Drouvot
> PostgreSQL Contributors Team
> RDS Open Source Databases
> Amazon Web Services: https://aws.amazon.com
>
Attachments: download all (3 files, .tar.gz)
v3-0001-fix-empty-subxact-serialized-flag.patch (text/x-patch, 4.7K) download
From 069ef9104702a73e4ac8881b178fc22d7d720657 Mon Sep 17 00:00:00 2001
From: Atish Prakash <125atishprak@gmail.com>
Date: Sun, 8 Nov 2026 21:06:11 +0530
Subject: [PATCH v3] Don't mark a transaction serialized when nothing was
spilled
ReorderBufferSerializeTXN() set RBTXN_IS_SERIALIZED on every transaction
it visited, including subtransactions with no changes to write. Such a
transaction keeps an invalid final_lsn, because final_lsn is only advanced
as changes are written, so ReorderBufferRestoreCleanup() can later fail its
assertion on it. This happens when ReorderBufferCheckAndTruncateAbortedTXN()
truncates an aborted transaction, and when a transaction is truncated after
being streamed once a partial change completes.
It also made spill_txns undercount: when such a subtransaction later spilled
real changes, it was not counted, as it was already marked serialized.
Set the flag only when changes were written, after the statistics update
that checks it.
Test case by Zhijie Hou.
Bug: #19839
Reported-by: Atish Prakash <125atishprak@gmail.com>
Author: Atish Prakash <125atishprak@gmail.com>
Reviewed-by: Zhijie Hou <houzhijie22@gmail.com>
Backpatch-through: 14
---
contrib/test_decoding/expected/stream.out | 24 +++++++++++++++++++
contrib/test_decoding/sql/stream.sql | 15 ++++++++++++
.../replication/logical/reorderbuffer.c | 5 ++--
3 files changed, 42 insertions(+), 2 deletions(-)
diff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.out
index 0ec5c93361..9c164847b0 100644
--- a/contrib/test_decoding/expected/stream.out
+++ b/contrib/test_decoding/expected/stream.out
@@ -157,6 +157,30 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
COMMIT
(3 rows)
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change
+-- (stream_test.data has EXTERNAL storage set above). Otherwise, truncating
+-- the transaction after streaming fails on the subtransaction's invalid
+-- final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+ ?column?
+----------
+ 1
+(1 row)
+
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+ count
+-------
+ 4
+(1 row)
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sql
index 5e45a8e9b6..f315933556 100644
--- a/contrib/test_decoding/sql/stream.sql
+++ b/contrib/test_decoding/sql/stream.sql
@@ -82,5 +82,20 @@ ROLLBACK;
INSERT INTO stream_test VALUES ('after-abort');
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change
+-- (stream_test.data has EXTERNAL storage set above). Otherwise, truncating
+-- the transaction after streaming fails on the subtransaction's invalid
+-- final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 4684398531..8af25c865c 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -4200,7 +4200,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* Update the memory counter */
ReorderBufferChangeMemoryUpdate(rb, NULL, txn, false, size);
- /* update the statistics iff we have spilled anything */
+ /* update the statistics and mark the transaction serialized iff we spilled */
if (spilled)
{
rb->spillCount += 1;
@@ -4211,12 +4211,13 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* update the decoding stats */
UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
+
+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
}
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
--
2.53.0
v3-REL_14_STABLE.patch (text/x-patch, 4.6K) download
From 337aad98b4d968a828a7de4e819ca2ec2121ef60 Mon Sep 17 00:00:00 2001
From: Atish Prakash <125atishprak@gmail.com>
Date: Thu, 12 Nov 2026 21:05:45 +0530
Subject: [PATCH REL_14_STABLE v3] Don't mark a transaction serialized when
nothing was spilled
ReorderBufferSerializeTXN() set RBTXN_IS_SERIALIZED on every transaction
it visited, including subtransactions with no changes to write. Such a
transaction keeps an invalid final_lsn, because final_lsn is only advanced
as changes are written, so ReorderBufferRestoreCleanup() can later fail its
assertion on it. This happens when ReorderBufferCheckAndTruncateAbortedTXN()
truncates an aborted transaction, and when a transaction is truncated after
being streamed once a partial change completes.
It also made spill_txns undercount: when such a subtransaction later spilled
real changes, it was not counted, as it was already marked serialized.
Set the flag only when changes were written, after the statistics update
that checks it.
Test case by Zhijie Hou.
Bug: #19839
Reported-by: Atish Prakash <125atishprak@gmail.com>
Author: Atish Prakash <125atishprak@gmail.com>
Reviewed-by: Zhijie Hou <houzhijie22@gmail.com>
Backpatch-through: 14
---
contrib/test_decoding/expected/stream.out | 23 +++++++++++++++++++
contrib/test_decoding/sql/stream.sql | 14 +++++++++++
.../replication/logical/reorderbuffer.c | 5 ++--
3 files changed, 40 insertions(+), 2 deletions(-)
diff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.out
index 0f21dcb8e0..0b6288fb14 100644
--- a/contrib/test_decoding/expected/stream.out
+++ b/contrib/test_decoding/expected/stream.out
@@ -106,6 +106,29 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL,NULL, 'incl
committing streamed transaction
(17 rows)
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change.
+-- Otherwise, truncating the transaction after streaming fails on the
+-- subtransaction's invalid final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+ ?column?
+----------
+ 1
+(1 row)
+
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+ count
+-------
+ 4
+(1 row)
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sql
index 4feec62972..1541e5dca3 100644
--- a/contrib/test_decoding/sql/stream.sql
+++ b/contrib/test_decoding/sql/stream.sql
@@ -44,5 +44,19 @@ toasted-123456789012345678901234567890123456789012345678901234567890123456789012
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL,NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change.
+-- Otherwise, truncating the transaction after streaming fails on the
+-- subtransaction's invalid final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 48a19ed4f6..cd712199d7 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3885,7 +3885,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
spilled++;
}
- /* update the statistics iff we have spilled anything */
+ /* update the statistics and mark the transaction serialized iff we spilled */
if (spilled)
{
rb->spillCount += 1;
@@ -3896,12 +3896,13 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* update the decoding stats */
UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
+
+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
}
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
--
2.53.0
v3-REL_17_STABLE.patch (text/x-patch, 4.6K) download
From c4fe4175e9d37e3aca7924077a8fcdf1fae38f16 Mon Sep 17 00:00:00 2001
From: Atish Prakash <125atishprak@gmail.com>
Date: Thu, 12 Nov 2026 21:05:45 +0530
Subject: [PATCH REL_17_STABLE v3] Don't mark a transaction serialized when
nothing was spilled
ReorderBufferSerializeTXN() set RBTXN_IS_SERIALIZED on every transaction
it visited, including subtransactions with no changes to write. Such a
transaction keeps an invalid final_lsn, because final_lsn is only advanced
as changes are written, so ReorderBufferRestoreCleanup() can later fail its
assertion on it. This happens when ReorderBufferCheckAndTruncateAbortedTXN()
truncates an aborted transaction, and when a transaction is truncated after
being streamed once a partial change completes.
It also made spill_txns undercount: when such a subtransaction later spilled
real changes, it was not counted, as it was already marked serialized.
Set the flag only when changes were written, after the statistics update
that checks it.
Test case by Zhijie Hou.
Bug: #19839
Reported-by: Atish Prakash <125atishprak@gmail.com>
Author: Atish Prakash <125atishprak@gmail.com>
Reviewed-by: Zhijie Hou <houzhijie22@gmail.com>
Backpatch-through: 14
---
contrib/test_decoding/expected/stream.out | 23 +++++++++++++++++++
contrib/test_decoding/sql/stream.sql | 14 +++++++++++
.../replication/logical/reorderbuffer.c | 5 ++--
3 files changed, 40 insertions(+), 2 deletions(-)
diff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.out
index a76f77601e..a4b51c0051 100644
--- a/contrib/test_decoding/expected/stream.out
+++ b/contrib/test_decoding/expected/stream.out
@@ -128,6 +128,29 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL,
5
(1 row)
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change.
+-- Otherwise, truncating the transaction after streaming fails on the
+-- subtransaction's invalid final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+ ?column?
+----------
+ 1
+(1 row)
+
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+ count
+-------
+ 4
+(1 row)
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sql
index 7f43f0c2ab..22a4e1255b 100644
--- a/contrib/test_decoding/sql/stream.sql
+++ b/contrib/test_decoding/sql/stream.sql
@@ -59,5 +59,19 @@ ROLLBACK TO s1;
COMMIT;
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+-- bug #19839
+--
+-- A subtransaction without decoded changes must not be marked as serialized
+-- when its parent transaction spills to disk during a partial TOAST change.
+-- Otherwise, truncating the transaction after streaming fails on the
+-- subtransaction's invalid final_lsn.
+BEGIN;
+SAVEPOINT s;
+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;
+RELEASE SAVEPOINT s;
+INSERT INTO stream_test VALUES (repeat('a', 100000));
+COMMIT;
+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 50989db6c4..5560020d7c 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -4038,7 +4038,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* Update the memory counter */
ReorderBufferChangeMemoryUpdate(rb, NULL, txn, false, size);
- /* update the statistics iff we have spilled anything */
+ /* update the statistics and mark the transaction serialized iff we spilled */
if (spilled)
{
rb->spillCount += 1;
@@ -4049,12 +4049,13 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* update the decoding stats */
UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
+
+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
}
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
--
2.53.0
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2026-11-12 15:40 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-07 01:23 BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction PG Bug reporting form <noreply@postgresql.org>
2026-11-07 01:26 ` Atisss hh <125atishprak@gmail.com>
2026-11-08 15:11 ` Zhijie Hou <houzhijie22@gmail.com>
2026-11-09 10:53 ` Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-11-09 11:46 ` Zhijie Hou <houzhijie22@gmail.com>
2026-11-09 14:47 ` Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-11-12 15:40 ` Atisss hh <125atishprak@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox