From: Zhijie Hou <houzhijie22@gmail.com>
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Cc: Atisss hh <125atishprak@gmail.com>
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
Date: Mon, 9 Nov 2026 19:46:59 +0800
Message-ID: <ba2ec30c-e381-4422-913b-24afb4263fc8@gmail.com> (raw)
In-Reply-To: <avGmDSdgyIZB73Sl@bdtpg>
References: <19839-ead02c2208a3f8bd@postgresql.org>
<CAGqfTtvQB4J0dJFnNMAF9LCAJhGBe5THd2R_=MxTHP5C5vwD=w@mail.gmail.com>
<1da45744-968f-4f40-addf-dc085e928ec5@gmail.com>
<avGmDSdgyIZB73Sl@bdtpg>
Hi,
On Mon, Nov 9, 2026 at 06:53 PM Bertrand Drouvot wrote:
>
> On Sun, Nov 08, 2026 at 11:11:09PM +0800, Zhijie Hou wrote:
> > Thanks for the report and patch.
>
> +1.
>
> > See the attachment for a test to reproduce this issue. (Feel free to merge
> > if it's worth adding a test to cover this path)
>
> > I think in a rare case, this can also affect the stats collection
> > for spilled txns. For example, if an empty subtransaction is marked
> > as
> > serialized, we skip incrementing spillTxns because no changes were written;
> > if real changes are later serialized in this subtransaction, we still do not
> > increment spillTxns because the subtransaction is already marked as
> > serialized and we avoid counting it "twice".
>
> Agreed.
>
> That's why I think the test is worth adding (as the bug could also
> produce real user visible impact as it can cause spill_txns to be
> undercounted on non assert enabled builds). The test only detects the
> regression on assert enabled builds, but I think that is enough. A
> test for the spill_txns case would be more complicated and is probably
> not worth it.
>
> >
> > + if (spilled)
> > + txn->txn_flags |= RBTXN_IS_SERIALIZED;
> >
> > ReorderBufferSerializeTXN() already has an "if (spilled)" block
> > right above this, so we shall directly move the assignment into that
> > block.
>
> +1.
Thanks for the review. It looks like Atisss's update patches didn't
reach the list, probably a missing Reply-All, so I'm helping to attach
the same patches (provided by Atisss) here.
--
Best Regards,
Zhijie Hou
Attachments:download all (3 files, .tar.gz)v2-0001-fix-empty-subxact-serialized-flag.patch (text/x-patch, 4.7K) download
From 092dc34678cd151de4262322bda1b2d31b1cec11 Mon Sep 17 00:00:00 2001
From: Atish Prakash <125atishprak@gmail.com>
Date: Sun, 8 Nov 2026 21:06:11 +0530
Subject: [PATCH v2] Don't mark a transaction serialized when nothing was
spilled
ReorderBufferSerializeTXN() set RBTXN_IS_SERIALIZED on every transaction
it visited, including subtransactions with no changes to write. Such a
transaction keeps an invalid final_lsn, because final_lsn is only advanced
as changes are written, so ReorderBufferRestoreCleanup() can later fail its
assertion on it. This happens when ReorderBufferCheckAndTruncateAbortedTXN()
truncates an aborted transaction, and when a transaction is truncated after
being streamed once a partial change completes.
It also made spill_txns undercount: when such a subtransaction later spilled
real changes, it was not counted, as it was already marked serialized.
Set the flag only when changes were written, after the statistics update
that checks it.
Test case by Zhijie Hou.
Bug: #19839
Reported-by: Atish Prakash <125atishprak@gmail.com>
Author: Atish Prakash <125atishprak@gmail.com>
Reviewed-by: Zhijie Hou <houzhijie22@gmail.com>
Backpatch-through: 14
---
contrib/test_decoding/expected/stream.out | 24 +++++++++++++++++++
contrib/test_decoding/sql/stream.sql | 15 ++++++++++++
.../replication/logical/reorderbuffer.c | 11 ++++++++-
3 files changed, 49 insertions(+), 1 deletion(-)
diff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.outindex 0ec5c93361..9c164847b0 100644--- a/contrib/test_decoding/expected/stream.out+++ b/contrib/test_decoding/expected/stream.out@@ -157,6 +157,30 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
COMMIT
(3 rows)
+-- bug #19839+--+-- A subtransaction without decoded changes must not be marked as serialized+-- when its parent transaction spills to disk during a partial TOAST change+-- (stream_test.data has EXTERNAL storage set above). Otherwise, truncating+-- the transaction after streaming fails on the subtransaction's invalid+-- final_lsn.+BEGIN;+SAVEPOINT s;+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;+ ?column? +----------+ 1+(1 row)++RELEASE SAVEPOINT s;+INSERT INTO stream_test VALUES (repeat('a', 100000));+COMMIT;+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');+ count +-------+ 4+(1 row)+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sqlindex 5e45a8e9b6..f315933556 100644--- a/contrib/test_decoding/sql/stream.sql+++ b/contrib/test_decoding/sql/stream.sql@@ -82,5 +82,20 @@ ROLLBACK;
INSERT INTO stream_test VALUES ('after-abort');
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+-- bug #19839+--+-- A subtransaction without decoded changes must not be marked as serialized+-- when its parent transaction spills to disk during a partial TOAST change+-- (stream_test.data has EXTERNAL storage set above). Otherwise, truncating+-- the transaction after streaming fails on the subtransaction's invalid+-- final_lsn.+BEGIN;+SAVEPOINT s;+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;+RELEASE SAVEPOINT s;+INSERT INTO stream_test VALUES (repeat('a', 100000));+COMMIT;+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.cindex 4684398531..acbd2c1344 100644--- a/src/backend/replication/logical/reorderbuffer.c+++ b/src/backend/replication/logical/reorderbuffer.c@@ -4211,12 +4211,21 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* update the decoding stats */
UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
++ /*+ * Mark the transaction as serialized only if something was written,+ * and only after the check above. final_lsn is advanced for each+ * change written, so a transaction that wrote nothing, such as a+ * subtransaction without decoded changes, would be marked with an+ * invalid final_lsn, and ReorderBufferRestoreCleanup() would later+ * fail on it.+ */+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
}
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
--
2.53.0
v2-REL_14_STABLE.patch (text/x-patch, 3.3K) downloaddiff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.outindex 0f21dcb8e0..0b6288fb14 100644--- a/contrib/test_decoding/expected/stream.out+++ b/contrib/test_decoding/expected/stream.out@@ -106,6 +106,29 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL,NULL, 'incl
committing streamed transaction
(17 rows)
+-- bug #19839+--+-- A subtransaction without decoded changes must not be marked as serialized+-- when its parent transaction spills to disk during a partial TOAST change.+-- Otherwise, truncating the transaction after streaming fails on the+-- subtransaction's invalid final_lsn.+BEGIN;+SAVEPOINT s;+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;+ ?column? +----------+ 1+(1 row)++RELEASE SAVEPOINT s;+INSERT INTO stream_test VALUES (repeat('a', 100000));+COMMIT;+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');+ count +-------+ 4+(1 row)+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sqlindex 4feec62972..1541e5dca3 100644--- a/contrib/test_decoding/sql/stream.sql+++ b/contrib/test_decoding/sql/stream.sql@@ -44,5 +44,19 @@ toasted-123456789012345678901234567890123456789012345678901234567890123456789012
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL,NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+-- bug #19839+--+-- A subtransaction without decoded changes must not be marked as serialized+-- when its parent transaction spills to disk during a partial TOAST change.+-- Otherwise, truncating the transaction after streaming fails on the+-- subtransaction's invalid final_lsn.+BEGIN;+SAVEPOINT s;+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;+RELEASE SAVEPOINT s;+INSERT INTO stream_test VALUES (repeat('a', 100000));+COMMIT;+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.cindex 48a19ed4f6..71bf1e8e3d 100644--- a/src/backend/replication/logical/reorderbuffer.c+++ b/src/backend/replication/logical/reorderbuffer.c@@ -3896,12 +3896,21 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* update the decoding stats */
UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
++ /*+ * Mark the transaction as serialized only if something was written,+ * and only after the check above. final_lsn is advanced for each+ * change written, so a transaction that wrote nothing, such as a+ * subtransaction without decoded changes, would be marked with an+ * invalid final_lsn, and ReorderBufferRestoreCleanup() would later+ * fail on it.+ */+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
}
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
v2-REL_17_STABLE.patch (text/x-patch, 3.2K) downloaddiff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.outindex a76f77601e..a4b51c0051 100644--- a/contrib/test_decoding/expected/stream.out+++ b/contrib/test_decoding/expected/stream.out@@ -128,6 +128,29 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL,
5
(1 row)
+-- bug #19839+--+-- A subtransaction without decoded changes must not be marked as serialized+-- when its parent transaction spills to disk during a partial TOAST change.+-- Otherwise, truncating the transaction after streaming fails on the+-- subtransaction's invalid final_lsn.+BEGIN;+SAVEPOINT s;+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;+ ?column? +----------+ 1+(1 row)++RELEASE SAVEPOINT s;+INSERT INTO stream_test VALUES (repeat('a', 100000));+COMMIT;+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');+ count +-------+ 4+(1 row)+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sqlindex 7f43f0c2ab..22a4e1255b 100644--- a/contrib/test_decoding/sql/stream.sql+++ b/contrib/test_decoding/sql/stream.sql@@ -59,5 +59,19 @@ ROLLBACK TO s1;
COMMIT;
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');
+-- bug #19839+--+-- A subtransaction without decoded changes must not be marked as serialized+-- when its parent transaction spills to disk during a partial TOAST change.+-- Otherwise, truncating the transaction after streaming fails on the+-- subtransaction's invalid final_lsn.+BEGIN;+SAVEPOINT s;+SELECT 1 FROM stream_test LIMIT 1 FOR UPDATE;+RELEASE SAVEPOINT s;+INSERT INTO stream_test VALUES (repeat('a', 100000));+COMMIT;+SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1');+
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.cindex 50989db6c4..1eb42b1396 100644--- a/src/backend/replication/logical/reorderbuffer.c+++ b/src/backend/replication/logical/reorderbuffer.c@@ -4049,12 +4049,21 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* update the decoding stats */
UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
++ /*+ * Mark the transaction as serialized only if something was written,+ * and only after the check above. final_lsn is advanced for each+ * change written, so a transaction that wrote nothing, such as a+ * subtransaction without decoded changes, would be marked with an+ * invalid final_lsn, and ReorderBufferRestoreCleanup() would later+ * fail on it.+ */+ txn->txn_flags |= RBTXN_IS_SERIALIZED;
}
Assert(spilled == txn->nentries_mem);
Assert(dlist_is_empty(&txn->changes));
txn->nentries_mem = 0;
- txn->txn_flags |= RBTXN_IS_SERIALIZED;
if (fd != -1)
CloseTransientFile(fd);
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-bugs@postgresql.org
Cc: houzhijie22@gmail.com, bertranddrouvot.pg@gmail.com, 125atishprak@gmail.com
Subject: Re: BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction
In-Reply-To: <ba2ec30c-e381-4422-913b-24afb4263fc8@gmail.com>
References: <19839-ead02c2208a3f8bd@postgresql.org> <CAGqfTtvQB4J0dJFnNMAF9LCAJhGBe5THd2R_=MxTHP5C5vwD=w@mail.gmail.com> <1da45744-968f-4f40-addf-dc085e928ec5@gmail.com> <avGmDSdgyIZB73Sl@bdtpg> <ba2ec30c-e381-4422-913b-24afb4263fc8@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox