agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v3 11/12] s/recommendable/recommended 486+ messages / 3 participants [nested] [flat]
* [PATCH v3 11/12] s/recommendable/recommended @ 2019-05-10 02:22 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Justin Pryzby @ 2019-05-10 02:22 UTC (permalink / raw) --- doc/src/sgml/btree.sgml | 2 +- doc/src/sgml/libpq.sgml | 2 +- doc/src/sgml/runtime.sgml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/btree.sgml b/doc/src/sgml/btree.sgml index 996932e..283db7f 100644 --- a/doc/src/sgml/btree.sgml +++ b/doc/src/sgml/btree.sgml @@ -60,7 +60,7 @@ contain the single-type operators (and associated support functions) for its input data type, while cross-type comparison operators and support functions are <quote>loose</quote> in the family. It is - recommendable that a complete set of cross-type operators be included + recommended that a complete set of cross-type operators be included in the family, thus ensuring that the planner can represent any comparison conditions that it deduces from transitivity. </para> diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 8a8427f..4b031ff 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -7107,7 +7107,7 @@ int PQresultSetInstanceData(PGresult *res, PGEventProc proc, void *data); Beware that any storage represented by <parameter>data</parameter> will not be accounted for by <function>PQresultMemorySize</function>, unless it is allocated using <function>PQresultAlloc</function>. - (Doing so is recommendable because it eliminates the need to free + (Doing so is recommended because it eliminates the need to free such storage explicitly when the result is destroyed.) </para> </listitem> diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml index 798da30..21a7ce3 100644 --- a/doc/src/sgml/runtime.sgml +++ b/doc/src/sgml/runtime.sgml @@ -111,7 +111,7 @@ <command>initdb</command> will attempt to create the directory you specify if it does not already exist. Of course, this will fail if <command>initdb</command> does not have permissions to write in the - parent directory. It's generally recommendable that the + parent directory. It's generally recommended that the <productname>PostgreSQL</productname> user own not just the data directory but its parent directory as well, so that this should not be a problem. If the desired parent directory doesn't exist either, -- 2.7.4 --cWoXeonUoKmBZSoM Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0012-Cleanup-remove-update-references-to-OID-column.patch" ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH v5 12/12] s/recommendable/recommended @ 2019-05-10 02:22 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Justin Pryzby @ 2019-05-10 02:22 UTC (permalink / raw) --- doc/src/sgml/btree.sgml | 2 +- doc/src/sgml/libpq.sgml | 2 +- doc/src/sgml/runtime.sgml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/btree.sgml b/doc/src/sgml/btree.sgml index 5881ea5..b0e0f08 100644 --- a/doc/src/sgml/btree.sgml +++ b/doc/src/sgml/btree.sgml @@ -60,7 +60,7 @@ contain the single-type operators (and associated support functions) for its input data type, while cross-type comparison operators and support functions are <quote>loose</quote> in the family. It is - recommendable that a complete set of cross-type operators be included + recommended that a complete set of cross-type operators be included in the family, thus ensuring that the planner can represent any comparison conditions that it deduces from transitivity. </para> diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 8a8427f..4b031ff 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -7107,7 +7107,7 @@ int PQresultSetInstanceData(PGresult *res, PGEventProc proc, void *data); Beware that any storage represented by <parameter>data</parameter> will not be accounted for by <function>PQresultMemorySize</function>, unless it is allocated using <function>PQresultAlloc</function>. - (Doing so is recommendable because it eliminates the need to free + (Doing so is recommended because it eliminates the need to free such storage explicitly when the result is destroyed.) </para> </listitem> diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml index ecdaafc..e3d0dec 100644 --- a/doc/src/sgml/runtime.sgml +++ b/doc/src/sgml/runtime.sgml @@ -111,7 +111,7 @@ <command>initdb</command> will attempt to create the directory you specify if it does not already exist. Of course, this will fail if <command>initdb</command> does not have permissions to write in the - parent directory. It's generally recommendable that the + parent directory. It's generally recommended that the <productname>PostgreSQL</productname> user own not just the data directory but its parent directory as well, so that this should not be a problem. If the desired parent directory doesn't exist either, -- 2.7.4 --FkmkrVfFsRoUs1wW-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH 1/2] Demonstrate possible race conditions in logical decoding. @ 2026-01-19 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Antonin Houska @ 2026-01-19 11:54 UTC (permalink / raw) The problem is that the snapshot builder can create a snapshot before CLOG has been updated. That breaks visibility check that use such snapshot. For more details, see startup_race.spec. Success of the test means that the problem is present. Thus it would need to be modified if it should be merged into the tree. Another problem that currently prevents this test from being merged is that it hard-wires the logical decoding setup into the SET TRANSACTION command. I tried to modify the isolation tester so it can use the logical replication protocol (in which case the test could use the "CREATE_REPLICATION_SNAPSHOT ... (SNAPSHOT 'use')" command), but the tester does things that are not compatible with that protocol (e.g. it sets the application_name parameter). --- .../test_decoding/expected/startup_race.out | 85 ++++++++++++ contrib/test_decoding/specs/startup_race.spec | 126 ++++++++++++++++++ src/backend/access/transam/xact.c | 6 + src/backend/replication/logical/snapbuild.c | 3 + src/backend/utils/time/snapmgr.c | 66 +++++++++ src/include/utils/snapmgr.h | 1 + 6 files changed, 287 insertions(+) create mode 100644 contrib/test_decoding/expected/startup_race.out create mode 100644 contrib/test_decoding/specs/startup_race.spec diff --git a/contrib/test_decoding/expected/startup_race.out b/contrib/test_decoding/expected/startup_race.out new file mode 100644 index 00000000000..597fa617831 --- /dev/null +++ b/contrib/test_decoding/expected/startup_race.out @@ -0,0 +1,85 @@ +Parsed test spec with 6 sessions + +starting permutation: s1_assign_xid s2_set_snapshot s3_assign_xid s1_rollback s3_rollback s4_do_changes s5_wake_up_full_snapshot s2_scan s2_rollback s5_wake_up_before_clog s6_check +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE b(i int); + +step s2_set_snapshot: + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; + <waiting ...> +step s3_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s1_rollback: + ROLLBACK; + +step s3_rollback: + ROLLBACK; + +step s4_do_changes: + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; + <waiting ...> +step s5_wake_up_full_snapshot: + SELECT injection_points_wakeup('snapbuild-full-snapshot'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_set_snapshot: <... completed> +step s2_scan: + TABLE a; + +i|j +-+- +1|1 +2|2 +(2 rows) + +step s2_rollback: + ROLLBACK; + +step s5_wake_up_before_clog: + SELECT injection_points_wakeup('before-clog-update'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_do_changes: <... completed> +step s6_check: + SELECT * FROM a ORDER BY i; + +i|j +-+- +1|1 +2|2 +(2 rows) + +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/startup_race.spec b/contrib/test_decoding/specs/startup_race.spec new file mode 100644 index 00000000000..8f67e07fa7a --- /dev/null +++ b/contrib/test_decoding/specs/startup_race.spec @@ -0,0 +1,126 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE b(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('snapbuild-full-snapshot', 'wait'); +} +# Use special, hard-wired snapshot name to set the initial snapshot from +# logical replication slot. +step s2_set_snapshot +{ + BEGIN READ ONLY ISOLATION LEVEL REPEATABLE READ; + SET TRANSACTION SNAPSHOT 'from_slot'; +} +# Perform the scan. +step s2_scan +{ + TABLE a; +} +step s2_rollback +{ + ROLLBACK; +} +teardown +{ + SELECT injection_points_detach('snapbuild-full-snapshot'); +} + +session s3 +step s3_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s3_rollback +{ + ROLLBACK; +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('before-clog-update', 'wait'); +} +step s4_do_changes +{ + INSERT INTO a(i, j) VALUES (3, 3); + UPDATE a SET j = j + 1 WHERE i = 1; + DELETE FROM a WHERE i = 2; +} +teardown +{ + SELECT injection_points_detach('before-clog-update'); +} + +session s5 +step s5_wake_up_full_snapshot +{ + SELECT injection_points_wakeup('snapbuild-full-snapshot'); +} +step s5_wake_up_before_clog +{ + SELECT injection_points_wakeup('before-clog-update'); +} + +session s6 +step s6_check +{ + SELECT * FROM a ORDER BY i; +} + +permutation +# Let the snapshot builder go through all the states. The problematic case +# happens in the FULL_SNAPSHOT. +s1_assign_xid +# This should leave the builder in BUILDING_SNAPSHOT, waiting for the active +# transaction to end. +s2_set_snapshot +# Make sure that s1_rollback does not allow going to CONSISTENT directly. +s3_assign_xid +# Let the builder proceed to FULL_SNAPSHOT. It should stop at the injection +# point 'snapbuild-full-snapshot'. +s1_rollback +# The transaction of s3 is not needed anymore, CONSISTENT should be the next +# stage. +s3_rollback +# Perform data changes before the snapshot builder triggers creation of the +# RUNNING_XACTS record. This will stop before setting transaction status in +# CLOG. +s4_do_changes +# Unblock the injection point so that the snapshot can finally be created. +s5_wake_up_full_snapshot +# Use the snapshot for a scan. The snapshot will consider s4 not running +# anymore, however CLOG is not aware of the commit yet. Thus +# HeapTupleSatisfiesMVCC will consider the transaction aborted. In particular, +# for UPDATE, if both xmax of the old version and xmin of the new version are +# considered aborted, so the effects of the UPDATE are lost +# altogether. Similarly, INSERT and DELETE have no effect because the xmin +# transaction of the inserted tuple and xmax of the deleted tuple are +# considered aborted +s2_scan +s2_rollback +# CLOG can be updated now. +s5_wake_up_before_clog +# Scan the table again using a new transaction, with a normal transaction +# snapshot. The results are still wrong due to hint bits set incorrectly. +s6_check + diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c857e23552f..2fea45b2fed 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -65,6 +65,7 @@ #include "utils/builtins.h" #include "utils/combocid.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/relmapper.h" @@ -1348,6 +1349,9 @@ RecordTransactionCommit(void) &RelcacheInitFileInval); wrote_xlog = (XactLastRecEnd != 0); + /* Load the injection point before entering the critical section */ + INJECTION_POINT_LOAD("before-clog-update"); + /* * If we haven't been assigned an XID yet, we neither can, nor do we want * to write a COMMIT record. @@ -1514,6 +1518,8 @@ RecordTransactionCommit(void) { XLogFlush(XactLastRecEnd); + INJECTION_POINT_CACHED("before-clog-update", NULL); + /* * Now we may update the CLOG, if we wrote a COMMIT record above */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 7f79621b57e..9b09dc8eac1 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1387,6 +1388,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn errdetail("Waiting for transactions (approximately %d) older than %u to end.", running->xcnt, running->nextXid)); + INJECTION_POINT("snapbuild-full-snapshot", NULL); + SnapBuildWaitSnapshot(running, running->nextXid); } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2e6197f5f35..f327b779004 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -110,10 +110,13 @@ #include "access/subtrans.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlogutils.h" #include "datatype/timestamp.h" #include "lib/pairingheap.h" #include "miscadmin.h" #include "port/pg_lfind.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/fd.h" #include "storage/predicate.h" #include "storage/proc.h" @@ -1421,6 +1424,17 @@ ImportSnapshot(const char *idstr) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ"))); + if (strcmp(idstr, "from_slot") == 0) + { + Snapshot snap; + + snap = create_test_snapshot(); + /* XXX sourcevxid shouldn't be needed in this special case */ + SetTransactionSnapshot(snap, NULL, MyProcPid, MyProc); + + return; + } + /* * Verify the identifier: only 0-9, A-F and hyphens are allowed. We do * this mainly to prevent reading arbitrary files. @@ -1969,3 +1983,55 @@ ResOwnerReleaseSnapshot(Datum res) { UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res)); } + +/* + * CreateReplicationSlot() with the CRS_USE_SNAPSHOT option would be useful + * for testing, but regression tests cannot speak both replication and query + * protocol at the same time. This function can be used instead to create a + * snapshot for special tests of logical replication. + */ +Snapshot +create_test_snapshot(void) +{ + const char *slotname = "test_slot"; + const char *plugin; + LogicalDecodingContext *ctx; + Snapshot snap; + + /* + * XXX Hard-wired values are fine for the special test that needs this + * function. + */ + plugin = "test_decoding"; + + Assert(!MyReplicationSlot); + + CheckLogicalDecodingRequirements(); + + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, + false, false, false); + + /* + * Ensure the logical decoding is enabled before initializing the + * logical decoding context. + */ + EnsureLogicalDecodingEnabled(); + Assert(IsLogicalDecodingEnabled()); + + ctx = CreateInitDecodingContext(plugin, NIL, true, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* build initial snapshot, might take a while */ + DecodingContextFindStartpoint(ctx); + + /* Do what the function is called for. */ + snap = SnapBuildInitialSnapshot(ctx->snapshot_builder); + snap = CopySnapshot(snap); + FreeDecodingContext(ctx); + + return snap; +} diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index b8c01a291a1..9f0d60ebc1b 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -123,4 +123,5 @@ extern Snapshot RestoreSnapshot(char *start_address); struct PGPROC; extern void RestoreTransactionSnapshot(Snapshot snapshot, struct PGPROC *source_pgproc); +extern Snapshot create_test_snapshot(void); #endif /* SNAPMGR_H */ -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 486+ messages in thread
* [PATCH v4] Track skipped vacuum and analyze activity per relation @ 2026-03-24 04:09 Yugo Nagata <[email protected]> 0 siblings, 0 replies; 486+ messages in thread From: Yugo Nagata @ 2026-03-24 04:09 UTC (permalink / raw) This commit adds eight fields to the relation statistics that track the last time vacuum or analyze has been attempted but skipped due to lock unavailability, along with their counts: - last_skipped_vacuum - last_skipped_autovacuum - last_skipped_analyze - last_skipped_autoanalyze - skipped_vacuum_count - skipped_autovacuum_count - skipped_analyze_count - skipped_autoanalyze_count These field can help users confirm that autovacuum is actively attempting to run on a table that has not been vacuumed or analyzed for a long time, and that the lack of progress is due to repeated skips rather than inactivity. --- doc/src/sgml/monitoring.sgml | 88 ++++++++++++++++ src/backend/catalog/system_views.sql | 8 ++ src/backend/commands/vacuum.c | 102 ++++++++++++++----- src/backend/utils/activity/pgstat_relation.c | 64 ++++++++++++ src/backend/utils/adt/pgstatfuncs.c | 24 +++++ src/include/catalog/pg_proc.dat | 32 ++++++ src/include/pgstat.h | 16 +++ src/test/regress/expected/rules.out | 24 +++++ 8 files changed, 335 insertions(+), 23 deletions(-) diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 08d5b824552..a9b579d87a9 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4387,6 +4387,16 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>last_skipped_vacuum</structfield> <type>timestamp with time zone</type> + </para> + <para> + Last time a manual vacuum on this table was attempted but skipped due to + lock unavailability (not counting <command>VACUUM FULL</command>) + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>last_autovacuum</structfield> <type>timestamp with time zone</type> @@ -4397,6 +4407,16 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>last_skipped_autovacuum</structfield> <type>timestamp with time zone</type> + </para> + <para> + Last time a vacuum on this table by the autovacuum daemon was attempted + but skipped due to lock unavailability + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>last_analyze</structfield> <type>timestamp with time zone</type> @@ -4406,6 +4426,16 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>last_skipped_analyze</structfield> <type>timestamp with time zone</type> + </para> + <para> + Last time a manual analyze on this table was attempted but skipped due to + lock unavailability + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>last_autoanalyze</structfield> <type>timestamp with time zone</type> @@ -4416,6 +4446,16 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>last_skipped_autoanalyze</structfield> <type>timestamp with time zone</type> + </para> + <para> + Last time at which an analyze on this table by the autovacuum was + attempted but skipped due to lock unavailability + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>vacuum_count</structfield> <type>bigint</type> @@ -4426,6 +4466,16 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>skipped_vacuum_count</structfield> <type>bigint</type> + </para> + <para> + Number of times manual vacuums on this table have been attempted but skipped + due to lock unavailability (not counting <command>VACUUM FULL</command>) + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>autovacuum_count</structfield> <type>bigint</type> @@ -4436,6 +4486,16 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>skipped_autovacuum_count</structfield> <type>bigint</type> + </para> + <para> + Number of times vacuums on this table by the autovacuum daemon have been + attempted but skipped due to lock unavailability + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>analyze_count</structfield> <type>bigint</type> @@ -4445,6 +4505,16 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>skipped_analyze_count</structfield> <type>bigint</type> + </para> + <para> + Number of times manual analyzes on this table have been attempted but + skipped due to lock unavailability + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>autoanalyze_count</structfield> <type>bigint</type> @@ -4455,6 +4525,16 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>skipped_autoanalyze_count</structfield> <type>bigint</type> + </para> + <para> + Number of times analyzes on this table by the autovacuum daemon have + been attempted but skipped due to lock unavailability + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>total_vacuum_time</structfield> <type>double precision</type> @@ -4510,6 +4590,14 @@ description | Waiting for a newly initialized WAL file to reach durable storage </tgroup> </table> + <note> + <para> + When a manual vacuum or analyze on a parent table in an inheritance or + partitioning hierarchy is skipped, the statistics are recorded only for + the parent table, not for its children. + </para> + </note> + </sect2> <sect2 id="monitoring-pg-stat-autovacuum-scores-view"> diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 73a1c1c4670..f509fc7876b 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -736,13 +736,21 @@ CREATE VIEW pg_stat_all_tables AS pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze, pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum, pg_stat_get_last_vacuum_time(C.oid) as last_vacuum, + pg_stat_get_last_skipped_vacuum_time(C.oid) as last_skipped_vacuum, pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum, + pg_stat_get_last_skipped_autovacuum_time(C.oid) as last_skipped_autovacuum, pg_stat_get_last_analyze_time(C.oid) as last_analyze, + pg_stat_get_last_skipped_analyze_time(C.oid) as last_skipped_analyze, pg_stat_get_last_autoanalyze_time(C.oid) as last_autoanalyze, + pg_stat_get_last_skipped_autoanalyze_time(C.oid) as last_skipped_autoanalyze, pg_stat_get_vacuum_count(C.oid) AS vacuum_count, + pg_stat_get_skipped_vacuum_count(C.oid) AS skipped_vacuum_count, pg_stat_get_autovacuum_count(C.oid) AS autovacuum_count, + pg_stat_get_skipped_autovacuum_count(C.oid) AS skipped_autovacuum_count, pg_stat_get_analyze_count(C.oid) AS analyze_count, + pg_stat_get_skipped_analyze_count(C.oid) AS skipped_analyze_count, pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count, + pg_stat_get_skipped_autoanalyze_count(C.oid) AS skipped_autoanalyze_count, pg_stat_get_total_vacuum_time(C.oid) AS total_vacuum_time, pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time, pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time, diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 99d0db82ed7..97cd5ea8d10 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -793,8 +793,25 @@ vacuum_open_relation(Oid relid, RangeVar *relation, uint32 options, rel = try_relation_open(relid, NoLock); else { + int flags = 0; rel = NULL; rel_lock = false; + + if ((options & VACOPT_VACUUM) != 0) + { + if (AmAutoVacuumWorkerProcess()) + flags |= PGSTAT_REPORT_SKIPPED_AUTOVACUUM; + else + flags |= PGSTAT_REPORT_SKIPPED_VACUUM; + } + if ((options & VACOPT_ANALYZE) != 0) + { + if (AmAutoVacuumWorkerProcess()) + flags |= PGSTAT_REPORT_SKIPPED_AUTOANALYZE; + else + flags |= PGSTAT_REPORT_SKIPPED_ANALYZE; + } + pgstat_report_skipped_vacuum_analyze(relid, flags); } /* if relation is opened, leave */ @@ -802,7 +819,7 @@ vacuum_open_relation(Oid relid, RangeVar *relation, uint32 options, return rel; /* - * Relation could not be opened, hence generate if possible a log + * Relation could not be opened hence generate if possible a log * informing on the situation. * * If the RangeVar is not defined, we do not have enough information to @@ -905,7 +922,6 @@ expand_vacuum_rel(VacuumRelation *vrel, MemoryContext vac_context, Form_pg_class classForm; bool include_children; bool is_partitioned_table; - int rvr_opts; /* * Since autovacuum workers supply OIDs when calling vacuum(), no @@ -918,29 +934,69 @@ expand_vacuum_rel(VacuumRelation *vrel, MemoryContext vac_context, * below, as well as find_all_inheritors's expectation that the caller * holds some lock on the starting relation. */ - rvr_opts = (options & VACOPT_SKIP_LOCKED) ? RVR_SKIP_LOCKED : 0; - relid = RangeVarGetRelidExtended(vrel->relation, - AccessShareLock, - rvr_opts, - NULL, NULL); - - /* - * If the lock is unavailable, emit the same log statement that - * vacuum_rel() and analyze_rel() would. - */ - if (!OidIsValid(relid)) + if (!(options & VACOPT_SKIP_LOCKED)) { - if (options & VACOPT_VACUUM) - ereport(WARNING, - (errcode(ERRCODE_LOCK_NOT_AVAILABLE), - errmsg("skipping vacuum of \"%s\" --- lock not available", - vrel->relation->relname))); - else - ereport(WARNING, - (errcode(ERRCODE_LOCK_NOT_AVAILABLE), - errmsg("skipping analyze of \"%s\" --- lock not available", + relid = RangeVarGetRelidExtended(vrel->relation, + AccessShareLock, + 0, NULL, NULL); + if (!OidIsValid(relid)) + return vacrels; + } + else + { + /* Get relid for reporting before taking a lock */ + relid = RangeVarGetRelid(vrel->relation, NoLock, false); + + if (!ConditionalLockRelationOid(relid, AccessShareLock)) + { + int flags = 0; + /* + * If the lock is unavailable, emit the same log statement that + * vacuum_rel() and analyze_rel() would. + */ + if (options & VACOPT_VACUUM) + ereport(WARNING, + (errcode(ERRCODE_LOCK_NOT_AVAILABLE), + errmsg("skipping vacuum of \"%s\" --- lock not available", vrel->relation->relname))); - return vacrels; + else + ereport(WARNING, + (errcode(ERRCODE_LOCK_NOT_AVAILABLE), + errmsg("skipping analyze of \"%s\" --- lock not available", + vrel->relation->relname))); + + if ((options & VACOPT_VACUUM) != 0) + flags |= PGSTAT_REPORT_SKIPPED_VACUUM; + if ((options & VACOPT_ANALYZE) != 0) + flags |= PGSTAT_REPORT_SKIPPED_ANALYZE; + + pgstat_report_skipped_vacuum_analyze(relid, flags); + + return vacrels; + } + + /* + * Now that we have the lock, probe to see if the relation really + * exists or not. + */ + if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relid))) + { + if (options & VACOPT_VACUUM) + ereport(WARNING, + (errcode(ERRCODE_UNDEFINED_TABLE), + errmsg("skipping vacuum of \"%s\" --- relation no longer exists", + vrel->relation->relname))); + else + ereport(WARNING, + (errcode(ERRCODE_UNDEFINED_TABLE), + errmsg("skipping analyze of \"%s\" --- relation no longer exists", + vrel->relation->relname))); + + /* Release useless lock */ + UnlockRelationOid(relid, AccessShareLock); + + return vacrels; + } } /* diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index b2ca28f83ba..532d9023f8c 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -17,12 +17,14 @@ #include "postgres.h" +#include "access/htup_details.h" #include "access/twophase_rmgr.h" #include "access/xact.h" #include "catalog/catalog.h" #include "utils/memutils.h" #include "utils/pgstat_internal.h" #include "utils/rel.h" +#include "utils/syscache.h" #include "utils/timestamp.h" @@ -367,6 +369,68 @@ pgstat_report_analyze(Relation rel, (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } +/* + * Report that the table was skipped during vacuum or/and analyze. + */ +void +pgstat_report_skipped_vacuum_analyze(Oid relid, int flags) +{ + PgStat_EntryRef *entry_ref; + PgStatShared_Relation *shtabentry; + PgStat_StatTabEntry *tabentry; + TimestampTz ts; + HeapTuple classTup; + bool isshared; + + if (!pgstat_track_counts || !flags) + return; + + classTup = SearchSysCache1(RELOID, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(classTup)) + return; /* somebody deleted the rel, forget it */ + isshared = ((Form_pg_class) GETSTRUCT(classTup))->relisshared; + ReleaseSysCache(classTup); + + /* Store the data in the table's hash table entry. */ + ts = GetCurrentTimestamp(); + + /* block acquiring lock for the same reason as pgstat_report_autovac() */ + entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION, + isshared ? InvalidOid : MyDatabaseId, + relid, false); + + shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats; + tabentry = &shtabentry->stats; + + if (flags & PGSTAT_REPORT_SKIPPED_VACUUM) + { + tabentry->last_skipped_vacuum_time = ts; + tabentry->skipped_vacuum_count++; + } + else if (flags & PGSTAT_REPORT_SKIPPED_AUTOVACUUM) + { + tabentry->last_skipped_autovacuum_time = ts; + tabentry->skipped_autovacuum_count++; + } + + if (flags & PGSTAT_REPORT_SKIPPED_ANALYZE) + { + tabentry->last_skipped_analyze_time = ts; + tabentry->skipped_analyze_count++; + } + else if (flags & PGSTAT_REPORT_SKIPPED_AUTOANALYZE) + { + tabentry->last_skipped_autoanalyze_time = ts; + tabentry->skipped_autoanalyze_count++; + } + + pgstat_unlock_entry(entry_ref); + + /* see pgstat_report_vacuum() */ + pgstat_flush_io(false); + (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); +} + /* * count a tuple insertion of n tuples */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 1408de387ea..90a8968faa0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -84,6 +84,18 @@ PG_STAT_GET_RELENTRY_INT64(mod_since_analyze) /* pg_stat_get_numscans */ PG_STAT_GET_RELENTRY_INT64(numscans) +/* pg_stat_get_skipped_analyze_count */ +PG_STAT_GET_RELENTRY_INT64(skipped_analyze_count) + +/* pg_stat_get_skipped_autoanalyze_count */ +PG_STAT_GET_RELENTRY_INT64(skipped_autoanalyze_count) + +/* pg_stat_get_skipped_autovacuum_count */ +PG_STAT_GET_RELENTRY_INT64(skipped_autovacuum_count) + +/* pg_stat_get_skipped_vacuum_count */ +PG_STAT_GET_RELENTRY_INT64(skipped_vacuum_count) + /* pg_stat_get_tuples_deleted */ PG_STAT_GET_RELENTRY_INT64(tuples_deleted) @@ -170,6 +182,18 @@ PG_STAT_GET_RELENTRY_TIMESTAMPTZ(last_vacuum_time) /* pg_stat_get_lastscan */ PG_STAT_GET_RELENTRY_TIMESTAMPTZ(lastscan) +/* pg_stat_get_last_skipped_analyze_time */ +PG_STAT_GET_RELENTRY_TIMESTAMPTZ(last_skipped_analyze_time) + +/* pg_stat_get_last_skipped_autoanalyze_time */ +PG_STAT_GET_RELENTRY_TIMESTAMPTZ(last_skipped_autoanalyze_time) + +/* pg_stat_get_last_skipped_autovacuum_time */ +PG_STAT_GET_RELENTRY_TIMESTAMPTZ(last_skipped_autovacuum_time) + +/* pg_stat_get_last_skipped_vacuum_time */ +PG_STAT_GET_RELENTRY_TIMESTAMPTZ(last_skipped_vacuum_time) + /* pg_stat_get_stat_reset_time */ PG_STAT_GET_RELENTRY_TIMESTAMPTZ(stat_reset_time) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index fa9ae79082b..32debb34863 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5680,6 +5680,38 @@ proargmodes => '{o,o,o,o,o,o,o,o,o,o}', proargnames => '{oid,score,xid_score,mxid_score,vacuum_score,vacuum_insert_score,analyze_score,do_vacuum,do_analyze,for_wraparound}', prosrc => 'pg_stat_get_autovacuum_scores' }, +{ oid => '8142', descr => 'statistics: last skipped vacuum time for a table', + proname => 'pg_stat_get_last_skipped_vacuum_time', provolatile => 's', + proparallel => 'r', prorettype => 'timestamptz', proargtypes => 'oid', + prosrc => 'pg_stat_get_last_skipped_vacuum_time' }, +{ oid => '8143', descr => 'statistics: last skipped auto vacuum time for a table', + proname => 'pg_stat_get_last_skipped_autovacuum_time', provolatile => 's', + proparallel => 'r', prorettype => 'timestamptz', proargtypes => 'oid', + prosrc => 'pg_stat_get_last_skipped_autovacuum_time' }, +{ oid => '8144', descr => 'statistics: last skipped analyze time for a table', + proname => 'pg_stat_get_last_skipped_analyze_time', provolatile => 's', + proparallel => 'r', prorettype => 'timestamptz', proargtypes => 'oid', + prosrc => 'pg_stat_get_last_skipped_analyze_time' }, +{ oid => '8145', descr => 'statistics: last skipped auto analyze time for a table', + proname => 'pg_stat_get_last_skipped_autoanalyze_time', provolatile => 's', + proparallel => 'r', prorettype => 'timestamptz', proargtypes => 'oid', + prosrc => 'pg_stat_get_last_skipped_autoanalyze_time' }, +{ oid => '8146', descr => 'statistics: number of skipped vacuum for a table', + proname => 'pg_stat_get_skipped_vacuum_count', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_skipped_vacuum_count' }, +{ oid => '8147', descr => 'statistics: number of skipped auto vacuum for a table', + proname => 'pg_stat_get_skipped_autovacuum_count', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_skipped_autovacuum_count' }, +{ oid => '8148', descr => 'statistics: number of skipped analyzes for a table', + proname => 'pg_stat_get_skipped_analyze_count', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_skipped_analyze_count' }, +{ oid => '8149', descr => 'statistics: number of skipped auto analyzes for a table', + proname => 'pg_stat_get_skipped_autoanalyze_count', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_skipped_autoanalyze_count' }, { oid => '1936', descr => 'statistics: currently active backend IDs', proname => 'pg_stat_get_backend_idset', prorows => '100', proretset => 't', provolatile => 's', proparallel => 'r', prorettype => 'int4', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index dfa2e837638..98941f953d1 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -479,6 +479,15 @@ typedef struct PgStat_StatTabEntry TimestampTz last_autoanalyze_time; /* autovacuum initiated */ PgStat_Counter autoanalyze_count; + TimestampTz last_skipped_vacuum_time; /* user initiated vacuum */ + PgStat_Counter skipped_vacuum_count; + TimestampTz last_skipped_autovacuum_time; /* autovacuum initiated */ + PgStat_Counter skipped_autovacuum_count; + TimestampTz last_skipped_analyze_time; /* user initiated */ + PgStat_Counter skipped_analyze_count; + TimestampTz last_skipped_autoanalyze_time; /* autovacuum initiated */ + PgStat_Counter skipped_autoanalyze_count; + PgStat_Counter total_vacuum_time; /* times in milliseconds */ PgStat_Counter total_autovacuum_time; PgStat_Counter total_analyze_time; @@ -703,6 +712,13 @@ extern void pgstat_report_analyze(Relation rel, PgStat_Counter livetuples, PgStat_Counter deadtuples, bool resetcounter, TimestampTz starttime); +/* flags for pgstat_flush_backend() */ +#define PGSTAT_REPORT_SKIPPED_VACUUM (1 << 0) /* vacuum is skipped */ +#define PGSTAT_REPORT_SKIPPED_ANALYZE (1 << 1) /* analyze is skipped */ +#define PGSTAT_REPORT_SKIPPED_AUTOVACUUM (1 << 2) /* autovacuum is skipped */ +#define PGSTAT_REPORT_SKIPPED_AUTOANALYZE (1 << 3) /* autoanalyze is skipped */ +extern void pgstat_report_skipped_vacuum_analyze(Oid relid, int flags); + /* * If stats are enabled, but pending data hasn't been prepared yet, call * pgstat_assoc_relation() to do so. See its comment for why this is done diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index a65a5bf0c4f..9b2075d3373 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1835,13 +1835,21 @@ pg_stat_all_tables| SELECT c.oid AS relid, pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze, pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum, pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum, + pg_stat_get_last_skipped_vacuum_time(c.oid) AS last_skipped_vacuum, pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum, + pg_stat_get_last_skipped_autovacuum_time(c.oid) AS last_skipped_autovacuum, pg_stat_get_last_analyze_time(c.oid) AS last_analyze, + pg_stat_get_last_skipped_analyze_time(c.oid) AS last_skipped_analyze, pg_stat_get_last_autoanalyze_time(c.oid) AS last_autoanalyze, + pg_stat_get_last_skipped_autoanalyze_time(c.oid) AS last_skipped_autoanalyze, pg_stat_get_vacuum_count(c.oid) AS vacuum_count, + pg_stat_get_skipped_vacuum_count(c.oid) AS skipped_vacuum_count, pg_stat_get_autovacuum_count(c.oid) AS autovacuum_count, + pg_stat_get_skipped_autovacuum_count(c.oid) AS skipped_autovacuum_count, pg_stat_get_analyze_count(c.oid) AS analyze_count, + pg_stat_get_skipped_analyze_count(c.oid) AS skipped_analyze_count, pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count, + pg_stat_get_skipped_autoanalyze_count(c.oid) AS skipped_autoanalyze_count, pg_stat_get_total_vacuum_time(c.oid) AS total_vacuum_time, pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time, pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time, @@ -2346,13 +2354,21 @@ pg_stat_sys_tables| SELECT relid, n_mod_since_analyze, n_ins_since_vacuum, last_vacuum, + last_skipped_vacuum, last_autovacuum, + last_skipped_autovacuum, last_analyze, + last_skipped_analyze, last_autoanalyze, + last_skipped_autoanalyze, vacuum_count, + skipped_vacuum_count, autovacuum_count, + skipped_autovacuum_count, analyze_count, + skipped_analyze_count, autoanalyze_count, + skipped_autoanalyze_count, total_vacuum_time, total_autovacuum_time, total_analyze_time, @@ -2401,13 +2417,21 @@ pg_stat_user_tables| SELECT relid, n_mod_since_analyze, n_ins_since_vacuum, last_vacuum, + last_skipped_vacuum, last_autovacuum, + last_skipped_autovacuum, last_analyze, + last_skipped_analyze, last_autoanalyze, + last_skipped_autoanalyze, vacuum_count, + skipped_vacuum_count, autovacuum_count, + skipped_autovacuum_count, analyze_count, + skipped_analyze_count, autoanalyze_count, + skipped_autoanalyze_count, total_vacuum_time, total_autovacuum_time, total_analyze_time, -- 2.43.0 --Multipart=_Mon__13_Apr_2026_17_05_51_+0900_T4n8olu8FNI1Va/O-- ^ permalink raw reply [nested|flat] 486+ messages in thread
end of thread, other threads:[~2026-03-24 04:09 UTC | newest] Thread overview: 486+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2019-05-10 02:22 [PATCH v5 12/12] s/recommendable/recommended Justin Pryzby <[email protected]> 2019-05-10 02:22 [PATCH v3 11/12] s/recommendable/recommended Justin Pryzby <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-01-19 11:54 [PATCH 1/2] Demonstrate possible race conditions in logical decoding. Antonin Houska <[email protected]> 2026-03-24 04:09 [PATCH v4] Track skipped vacuum and analyze activity per relation Yugo Nagata <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox